Project Part 2

Team Name: Group 11

Project Title : Trend Minder Stock Predictor (TMSP)

Team Members: Amit Sai Jitta and Hemraj Yadav

Data Source

Yahoo Finance: https://en.wikipedia.org/wiki/List_of_S%26P_500_companies

Dataset Name : S&P 500 index

Author : Wikipedia

TEMPORAL COVERAGE START DATE : 2020

About Dataset

Dataset:

1569753 entries, 2020-01-02 to 2024-02-29

Our project dataset contains 1569753 rows with comprehensive financial information for companies listed on the S&P 500 index over a period of four years. It includes data points such as stock prices, trading volume, companies’ names etc.

Scope: All companies listed on the S&P 500 index

Duration: 4 years

Frequency: Daily (or as per your dataset's granularity)

Data Format: Tabular data, likely organized in a structured format such as CSV (Comma Separated Values)

Variables:

Date: specific calendar date on which a stock market transaction occurred, Data Type - Date (mm-dd-yyyy)

Open: price at which a particular stock begins trading on a given trading day, Data Type - floating

Close: final price at which a particular stock is traded on a given trading day, Data Type - floating

high: highest price at which a particular stock traded during a given trading day, Data Type - floating.

low: lowest price at which a particular stock traded during a given trading day, Data Type- floating.

Volume: total number of shares of a particular stock that are traded during a specified period, Data Type - Integer

Name: Unique identifier or symbol used to represent a particular publicly traded company on a stock exchange, Data Type – Character

Origin and Purpose:

Wiki collected the dataset with the purpose of conducting intriguing visualizations on stock market volatility. It has been made available for others to delve into the intricacies of the stock market domain and to engage in various data visualization exercises. One can look at simple things like how prices change over time, graph, and compare multiple stocks at once, or generate and graph new metrics from the data provided.From these data informative stock stats such as volatility and moving averages can be easily calculated.

Part 2: Design, data processing, and modeling

Data Collection:

Gather relevant time series data from appropriate sources. Ensure data quality and consistency.

Data Preprocessing:

Handle missing values, outliers, and noise in the data. Perform any necessary data transformations (e.g., normalization, scaling). Consider techniques like detrending or deseasonalizing if needed.

Exploratory Data Analysis (EDA):

Understand the underlying patterns, trends, and seasonality in the data. Visualize the time series data to gain insights. Identify potential relationships and correlations with other variables.

Feature Engineering:

Derive relevant features from the time series data. Consider lagged variables, rolling statistics, or domain-specific features. Feature selection may be necessary to reduce dimensionality.

Model Selection:

Choose appropriate time series models based on the problem and data characteristics. Consider traditional statistical models (e.g., ARIMA, SARIMA) or machine learning models (e.g., LSTM, Prophet). Evaluate the trade-offs between complexity and interpretability.

Model Training:

Split the data into training and validation/test sets. Train the selected models using the training data. Tune model hyperparameters to optimize performance.

Model Evaluation:

Evaluate model performance using appropriate metrics (e.g., Mean Absolute Error, Root Mean Squared Error). Compare the performance of different models. Validate the models using the validation/test set.

Model Interpretation:

Interpret the results of the trained models. Understand the impact of different features on the predictions. Identify any limitations or assumptions of the models.

Deployment and Monitoring:

Deploy the chosen model into production if applicable. Set up monitoring systems to track model performance over time. Implement feedback loops to update the model as new data becomes available.

Documentation and Reporting:

Document the entire process including data sources, preprocessing steps, model selection, and evaluation. Prepare a report summarizing the findings and recommendations. Communicate results effectively to stakeholders.

Iterative Improvement:

Continuously refine the model based on feedback and new data. Explore advanced techniques or alternative models to improve performance.

Environment Preperation

from google.colab import drive
drive.mount('/content/drive')

!pip install fasteda -q

  Preparing metadata (setup.py) ... done
  Building wheel for fasteda (setup.py) ... done

More information on the fasteda package.

import pandas as pd
import missingno as msno
import matplotlib.pyplot as plt
import seaborn as sns
import os
import numpy as np
from fasteda import fast_eda
import plotly.express as px
from statsmodels.tsa.seasonal import seasonal_decompose
from statsmodels.tsa.stattools import adfuller
from statsmodels.tsa.stattools import acf
from statsmodels.tsa.statespace.sarimax import SARIMAX
from statsmodels.tsa.holtwinters import ExponentialSmoothing
from statsmodels.stats.diagnostic import acorr_ljungbox
from scipy.stats import shapiro
from sklearn.metrics import mean_absolute_error, mean_squared_error
from pandas import read_csv
from matplotlib import pyplot
from pandas import Series
import warnings
from pandas import DataFrame
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.graphics.tsaplots import plot_acf
from statsmodels.graphics.tsaplots import plot_pacf
from scipy.stats import boxcox
from statsmodels.graphics.gofplots import qqplot
from sklearn.metrics import mean_squared_error
from math import sqrt
from math import exp
from math import log
import numpy
from statsmodels.tsa.arima.model import ARIMAResults
import yfinance as yf

Load Data

pip install yfinance

Requirement already satisfied: yfinance in /usr/local/lib/python3.10/dist-packages (0.2.37)
Requirement already satisfied: pandas>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.5.3)
Requirement already satisfied: numpy>=1.16.5 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.25.2)
Requirement already satisfied: requests>=2.31 in /usr/local/lib/python3.10/dist-packages (from yfinance) (2.31.0)
Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.10/dist-packages (from yfinance) (0.0.11)
Requirement already satisfied: lxml>=4.9.1 in /usr/local/lib/python3.10/dist-packages (from yfinance) (4.9.4)
Requirement already satisfied: appdirs>=1.4.4 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.4.4)
Requirement already satisfied: pytz>=2022.5 in /usr/local/lib/python3.10/dist-packages (from yfinance) (2023.4)
Requirement already satisfied: frozendict>=2.3.4 in /usr/local/lib/python3.10/dist-packages (from yfinance) (2.4.0)
Requirement already satisfied: peewee>=3.16.2 in /usr/local/lib/python3.10/dist-packages (from yfinance) (3.17.1)
Requirement already satisfied: beautifulsoup4>=4.11.1 in /usr/local/lib/python3.10/dist-packages (from yfinance) (4.12.3)
Requirement already satisfied: html5lib>=1.1 in /usr/local/lib/python3.10/dist-packages (from yfinance) (1.1)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4>=4.11.1->yfinance) (2.5)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.10/dist-packages (from html5lib>=1.1->yfinance) (1.16.0)
Requirement already satisfied: webencodings in /usr/local/lib/python3.10/dist-packages (from html5lib>=1.1->yfinance) (0.5.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas>=1.3.0->yfinance) (2.8.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.31->yfinance) (2024.2.2)

# Download S&P 500 data from Wikipedia
def get_sp500_tickers():

    sp500_df = pd.read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies")[0]
    sp500_tickers = sp500_df['Symbol'].tolist()

    return sp500_tickers

sp500_tickers = get_sp500_tickers()
print(sp500_tickers)

['MMM', 'AOS', 'ABT', 'ABBV', 'ACN', 'ADBE', 'AMD', 'AES', 'AFL', 'A', 'APD', 'ABNB', 'AKAM', 'ALB', 'ARE', 'ALGN', 'ALLE', 'LNT', 'ALL', 'GOOGL', 'GOOG', 'MO', 'AMZN', 'AMCR', 'AEE', 'AAL', 'AEP', 'AXP', 'AIG', 'AMT', 'AWK', 'AMP', 'AME', 'AMGN', 'APH', 'ADI', 'ANSS', 'AON', 'APA', 'AAPL', 'AMAT', 'APTV', 'ACGL', 'ADM', 'ANET', 'AJG', 'AIZ', 'T', 'ATO', 'ADSK', 'ADP', 'AZO', 'AVB', 'AVY', 'AXON', 'BKR', 'BALL', 'BAC', 'BK', 'BBWI', 'BAX', 'BDX', 'BRK.B', 'BBY', 'BIO', 'TECH', 'BIIB', 'BLK', 'BX', 'BA', 'BKNG', 'BWA', 'BXP', 'BSX', 'BMY', 'AVGO', 'BR', 'BRO', 'BF.B', 'BLDR', 'BG', 'CDNS', 'CZR', 'CPT', 'CPB', 'COF', 'CAH', 'KMX', 'CCL', 'CARR', 'CTLT', 'CAT', 'CBOE', 'CBRE', 'CDW', 'CE', 'COR', 'CNC', 'CNP', 'CF', 'CHRW', 'CRL', 'SCHW', 'CHTR', 'CVX', 'CMG', 'CB', 'CHD', 'CI', 'CINF', 'CTAS', 'CSCO', 'C', 'CFG', 'CLX', 'CME', 'CMS', 'KO', 'CTSH', 'CL', 'CMCSA', 'CMA', 'CAG', 'COP', 'ED', 'STZ', 'CEG', 'COO', 'CPRT', 'GLW', 'CTVA', 'CSGP', 'COST', 'CTRA', 'CCI', 'CSX', 'CMI', 'CVS', 'DHR', 'DRI', 'DVA', 'DAY', 'DE', 'DAL', 'XRAY', 'DVN', 'DXCM', 'FANG', 'DLR', 'DFS', 'DG', 'DLTR', 'D', 'DPZ', 'DOV', 'DOW', 'DHI', 'DTE', 'DUK', 'DD', 'EMN', 'ETN', 'EBAY', 'ECL', 'EIX', 'EW', 'EA', 'ELV', 'LLY', 'EMR', 'ENPH', 'ETR', 'EOG', 'EPAM', 'EQT', 'EFX', 'EQIX', 'EQR', 'ESS', 'EL', 'ETSY', 'EG', 'EVRG', 'ES', 'EXC', 'EXPE', 'EXPD', 'EXR', 'XOM', 'FFIV', 'FDS', 'FICO', 'FAST', 'FRT', 'FDX', 'FIS', 'FITB', 'FSLR', 'FE', 'FI', 'FLT', 'FMC', 'F', 'FTNT', 'FTV', 'FOXA', 'FOX', 'BEN', 'FCX', 'GRMN', 'IT', 'GEHC', 'GEN', 'GNRC', 'GD', 'GE', 'GIS', 'GM', 'GPC', 'GILD', 'GPN', 'GL', 'GS', 'HAL', 'HIG', 'HAS', 'HCA', 'DOC', 'HSIC', 'HSY', 'HES', 'HPE', 'HLT', 'HOLX', 'HD', 'HON', 'HRL', 'HST', 'HWM', 'HPQ', 'HUBB', 'HUM', 'HBAN', 'HII', 'IBM', 'IEX', 'IDXX', 'ITW', 'ILMN', 'INCY', 'IR', 'PODD', 'INTC', 'ICE', 'IFF', 'IP', 'IPG', 'INTU', 'ISRG', 'IVZ', 'INVH', 'IQV', 'IRM', 'JBHT', 'JBL', 'JKHY', 'J', 'JNJ', 'JCI', 'JPM', 'JNPR', 'K', 'KVUE', 'KDP', 'KEY', 'KEYS', 'KMB', 'KIM', 'KMI', 'KLAC', 'KHC', 'KR', 'LHX', 'LH', 'LRCX', 'LW', 'LVS', 'LDOS', 'LEN', 'LIN', 'LYV', 'LKQ', 'LMT', 'L', 'LOW', 'LULU', 'LYB', 'MTB', 'MRO', 'MPC', 'MKTX', 'MAR', 'MMC', 'MLM', 'MAS', 'MA', 'MTCH', 'MKC', 'MCD', 'MCK', 'MDT', 'MRK', 'META', 'MET', 'MTD', 'MGM', 'MCHP', 'MU', 'MSFT', 'MAA', 'MRNA', 'MHK', 'MOH', 'TAP', 'MDLZ', 'MPWR', 'MNST', 'MCO', 'MS', 'MOS', 'MSI', 'MSCI', 'NDAQ', 'NTAP', 'NFLX', 'NEM', 'NWSA', 'NWS', 'NEE', 'NKE', 'NI', 'NDSN', 'NSC', 'NTRS', 'NOC', 'NCLH', 'NRG', 'NUE', 'NVDA', 'NVR', 'NXPI', 'ORLY', 'OXY', 'ODFL', 'OMC', 'ON', 'OKE', 'ORCL', 'OTIS', 'PCAR', 'PKG', 'PANW', 'PARA', 'PH', 'PAYX', 'PAYC', 'PYPL', 'PNR', 'PEP', 'PFE', 'PCG', 'PM', 'PSX', 'PNW', 'PXD', 'PNC', 'POOL', 'PPG', 'PPL', 'PFG', 'PG', 'PGR', 'PLD', 'PRU', 'PEG', 'PTC', 'PSA', 'PHM', 'QRVO', 'PWR', 'QCOM', 'DGX', 'RL', 'RJF', 'RTX', 'O', 'REG', 'REGN', 'RF', 'RSG', 'RMD', 'RVTY', 'RHI', 'ROK', 'ROL', 'ROP', 'ROST', 'RCL', 'SPGI', 'CRM', 'SBAC', 'SLB', 'STX', 'SRE', 'NOW', 'SHW', 'SPG', 'SWKS', 'SJM', 'SNA', 'SO', 'LUV', 'SWK', 'SBUX', 'STT', 'STLD', 'STE', 'SYK', 'SYF', 'SNPS', 'SYY', 'TMUS', 'TROW', 'TTWO', 'TPR', 'TRGP', 'TGT', 'TEL', 'TDY', 'TFX', 'TER', 'TSLA', 'TXN', 'TXT', 'TMO', 'TJX', 'TSCO', 'TT', 'TDG', 'TRV', 'TRMB', 'TFC', 'TYL', 'TSN', 'USB', 'UBER', 'UDR', 'ULTA', 'UNP', 'UAL', 'UPS', 'URI', 'UNH', 'UHS', 'VLO', 'VTR', 'VLTO', 'VRSN', 'VRSK', 'VZ', 'VRTX', 'VFC', 'VTRS', 'VICI', 'V', 'VMC', 'WRB', 'WAB', 'WBA', 'WMT', 'DIS', 'WBD', 'WM', 'WAT', 'WEC', 'WFC', 'WELL', 'WST', 'WDC', 'WRK', 'WY', 'WHR', 'WMB', 'WTW', 'GWW', 'WYNN', 'XEL', 'XYL', 'YUM', 'ZBRA', 'ZBH', 'ZION', 'ZTS']

# Replace specified tickers with their replacements

tickers_to_replace = {"BF.B": "BF-B", "BRK.B": "BRK-B"}
for old_ticker, new_ticker in tickers_to_replace.items():
    if old_ticker in sp500_tickers:
        index = sp500_tickers.index(old_ticker)
        sp500_tickers[index] = new_ticker

# Loop through each ticker and retrieve historical data
for ticker in sp500_tickers:
    data = yf.download(ticker, start="2020-01-01", end="2024-03-01")
    data.to_csv("/content/drive/MyDrive/data/tmsp2/" + f"{ticker}_historical_data.csv")

[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed
[*********************100%%**********************]  1 of 1 completed

# Combined the data containing the individual CSV files to a new CSV file
directory_path = "/content/drive/MyDrive/data/tmsp2/"
combined_data = pd.DataFrame()

for filename in os.listdir(directory_path):
    if filename.endswith(".csv"):
        filepath = os.path.join(directory_path, filename)
        data = pd.read_csv(filepath)
        ticker = filename.split("_")[0]
        data["Ticker"] = ticker
        combined_data = combined_data.append(data, ignore_index=True)

combined_data.to_csv("/content/drive/MyDrive/data/tmsp2/combined_sp500_data.csv", index=False)

<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)
<ipython-input-6-9a36bd1b7834>:22: FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
  combined_data = combined_data.append(data, ignore_index=True)

# Display the first few rows of the DataFrame
combined_csv_path = "/content/drive/MyDrive/data/tmsp2/combined_sp500_data.csv"
combined_data = pd.read_csv(combined_csv_path)

print(combined_data.head())

         Date        Open        High         Low       Close   Adj Close  \
0  2020-01-02  177.679993  180.009995  177.139999  180.000000  149.672821   
1  2020-01-03  177.020004  178.660004  175.630005  178.449997  148.383957   
2  2020-01-06  177.149994  178.710007  176.350006  178.619995  148.525299   
3  2020-01-07  178.279999  178.509995  176.820007  177.899994  147.926605   
4  2020-01-08  178.000000  181.500000  177.649994  180.630005  150.196655   

    Volume Ticker  
0  3601700    MMM  
1  2466900    MMM  
2  1998000    MMM  
3  2173000    MMM  
4  2758300    MMM  

combined_data['date']=pd.to_datetime(combined_data['Date'])

combined_data['Date'] = pd.to_datetime(combined_data['Date'])
combined_data.set_index('Date', inplace=True)

# Display the summary information of the DataFrame
print("Summary Information:")
print(combined_data.info())
print("\nSummary Statistics:")
print(combined_data.describe())

Summary Information:
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 1569753 entries, 2020-01-02 to 2024-02-29
Data columns (total 8 columns):
 #   Column     Non-Null Count    Dtype         
---  ------     --------------    -----         
 0   Open       1569753 non-null  float64       
 1   High       1569753 non-null  float64       
 2   Low        1569753 non-null  float64       
 3   Close      1569753 non-null  float64       
 4   Adj Close  1569753 non-null  float64       
 5   Volume     1569753 non-null  int64         
 6   Ticker     1569753 non-null  object        
 7   date       1569753 non-null  datetime64[ns]
dtypes: datetime64[ns](1), float64(5), int64(1), object(1)
memory usage: 107.8+ MB
None

Summary Statistics:
               Open          High           Low         Close     Adj Close  \
count  1.569753e+06  1.569753e+06  1.569753e+06  1.569753e+06  1.569753e+06   
mean   1.646755e+02  1.667576e+02  1.625807e+02  1.647067e+02  1.605460e+02   
std    2.926456e+02  2.963269e+02  2.891349e+02  2.928045e+02  2.923336e+02   
min    3.220000e+00  3.290000e+00  3.020000e+00  3.120000e+00  2.957920e+00   
25%    5.543000e+01  5.617000e+01  5.468000e+01  5.542000e+01  5.218929e+01   
50%    1.008500e+02  1.022100e+02  9.951000e+01  1.008700e+02  9.659942e+01   
75%    1.903600e+02  1.927500e+02  1.879800e+02  1.904300e+02  1.843927e+02   
max    7.612250e+03  7.700000e+03  7.570100e+03  7.625570e+03  7.625570e+03   

             Volume  
count  1.569753e+06  
mean   5.366149e+06  
std    1.306298e+07  
min    0.000000e+00  
25%    9.724060e+05  
50%    2.034200e+06  
75%    4.665700e+06  
max    9.140820e+08  

# Check for null values in the DataFrame
null_values = combined_data.isnull().sum()
print("Count of Null Values:")
print(null_values)
if null_values.sum() == 0:
    print("\nNo null values found in the DataFrame.")
else:
    print("\nThere are null values in the DataFrame.")

Count of Null Values:
Open         0
High         0
Low          0
Close        0
Adj Close    0
Volume       0
Ticker       0
date         0
dtype: int64

No null values found in the DataFrame.

# Check for data type in the DataFrame
num_rows, num_cols = null_values = combined_data.shape
print("\nNumber of Rows:", num_rows)
print("Number of Columns:", num_cols)

combined_data.dtypes

Number of Rows: 1569753
Number of Columns: 8

Open                float64
High                float64
Low                 float64
Close               float64
Adj Close           float64
Volume                int64
Ticker               object
date         datetime64[ns]
dtype: object

# Find top 4 stocks based on mean market capitalization
ticker_market_caps = combined_data.groupby('Ticker')['Volume'].mean()
sorted_tickers = ticker_market_caps.sort_values(ascending=False)
stock_list = sorted_tickers.head(4)
print("Top 4 Stocks:")
print(stock_list)

Top 4 Stocks:
Ticker
TSLA    1.321701e+08
AAPL    9.732060e+07
AMZN    7.433595e+07
F       7.101031e+07
Name: Volume, dtype: float64

# Top_4_stocks based on trading volume
stock_list = stock_list.index.tolist()
print("Top 4 Tickers:")
print(stock_list)

Top 4 Tickers:
['TSLA', 'AAPL', 'AMZN', 'F']

Visualize your data

# Create subplots for multiple visualizations
fig, axes = plt.subplots(nrows=2, ncols=2, figsize=(12, 8))
combined_data[['Open', 'High', 'Low', 'Close']].plot(ax=axes[0, 0])
axes[0, 0].set_title('Open, High, Low, Close Prices Over Time')

# Plot Volume over time
combined_data['Volume'].plot(ax=axes[0, 1])
axes[0, 1].set_title('Volume Over Time')

# Boxplot for Open, High, Low, and Close Prices
sns.boxplot(data=combined_data[['Open', 'High', 'Low', 'Close']], ax=axes[1, 0])
axes[1, 0].set_title('Boxplot for Open, High, Low, Close Prices')

# Boxplot for Volume
sns.boxplot(data=combined_data['Volume'], ax=axes[1, 1])
axes[1, 1].set_title('Boxplot for Volume')

# Adjust layout for better visibility
plt.tight_layout()

# Show the plots
plt.show()

# Compute the correlation matrix
corr_matrix = combined_data[['Open', 'High', 'Low', 'Close', 'Volume']].corr()
sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', linewidths=0.5)
plt.title('Correlation Heatmap')
plt.show()

# Create a histogram for 'close' prices
plt.hist(combined_data['Close'], bins=30, color='blue', alpha=0.7)
plt.title('Histogram of Close Prices')
plt.xlabel('Close Price')
plt.ylabel('Frequency')
plt.show()

# Create a density plot for 'close' prices
sns.kdeplot(combined_data['Close'], fill=True, color='green')
plt.title('Density Plot of Close Prices')
plt.xlabel('Close Price')
plt.ylabel('Density')
plt.show()

Fast EDA

fast_eda(combined_data)

DataFrame Head:

	Open	High	Low	Close	Adj Close	Volume	Ticker	date
Date
2020-01-02	177.679993	180.009995	177.139999	180.000000	149.672821	3601700	MMM	2020-01-02
2020-01-03	177.020004	178.660004	175.630005	178.449997	148.383957	2466900	MMM	2020-01-03
2020-01-06	177.149994	178.710007	176.350006	178.619995	148.525299	1998000	MMM	2020-01-06

DataFrame Tail:

	Open	High	Low	Close	Adj Close	Volume	Ticker	date
Date
2024-02-27	197.649994	199.250000	194.039993	199.000000	199.000000	1982500	combined	2024-02-27
2024-02-28	199.300003	200.500000	197.100006	199.940002	199.940002	2003300	combined	2024-02-28
2024-02-29	200.500000	200.529999	198.190002	198.330002	198.330002	2849200	combined	2024-02-29

----------------------------------------------------------------------------------------------------
Missing values:

	0

----------------------------------------------------------------------------------------------------
Shape of DataFrame:

(1046502, 8)

----------------------------------------------------------------------------------------------------
DataFrame Info:

<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 1046502 entries, 2020-01-02 to 2024-02-29
Data columns (total 8 columns):
 #   Column     Non-Null Count    Dtype         
---  ------     --------------    -----         
 0   Open       1046502 non-null  float64       
 1   High       1046502 non-null  float64       
 2   Low        1046502 non-null  float64       
 3   Close      1046502 non-null  float64       
 4   Adj Close  1046502 non-null  float64       
 5   Volume     1046502 non-null  int64         
 6   Ticker     1046502 non-null  object        
 7   date       1046502 non-null  datetime64[ns]
dtypes: datetime64[ns](1), float64(5), int64(1), object(1)
memory usage: 71.9+ MB
----------------------------------------------------------------------------------------------------
Describe DataFrame:

	count	mean	median	std	min	25%	50%	75%	max	skewness	kurtosis
Open	1046502	164.675	100.85	292.646	3.22	55.43	100.85	190.36	7612.25	11.235268	179.581375
High	1046502	166.758	102.21	296.327	3.29	56.17	102.21	192.75	7700	11.230501	179.236282
Low	1046502	162.581	99.51	289.135	3.02	54.68	99.51	187.98	7570.1	11.251279	180.274746
Close	1046502	164.707	100.87	292.804	3.12	55.42	100.87	190.43	7625.57	11.241679	179.816582
Adj Close	1046502	160.546	96.599	292.334	2.958	52.189	96.599	184.393	7625.57	11.318520	181.535487
Volume	1046502	5366148.547	2034200	13062978.959	0	972410.25	2034200	4665675	914082000	11.007764	258.012613

----------------------------------------------------------------------------------------------------
DataFrame Correlation:

/usr/local/lib/python3.10/dist-packages/fasteda/__init__.py:124: FutureWarning: The default value of numeric_only in DataFrame.corr is deprecated. In a future version, it will default to False. Select only valid columns or specify the value of numeric_only to silence this warning.
  sns.heatmap(df.corr(), annot = True, cmap = "Spectral", linewidths = 2, linecolor = "#000000", fmt='.3f')

----------------------------------------------------------------------------------------------------
DataFrame Pairplot:

----------------------------------------------------------------------------------------------------
Histogram(s) & Boxplot(s):

Time Series Visualization

# Time Series of sortlisted stocks for Close
plt.figure(figsize=(20,12))
for i, company in enumerate(stock_list,1):
    plt.subplot(2, 2, i)
    df=combined_data[combined_data['Ticker']==company]
    plt.plot(df['date'],df['Close'])
    plt.title(company)

# Time Series of sortlisted stocks for Adj Close
plt.figure(figsize=(20,12))
for i, company in enumerate(stock_list,1):
    plt.subplot(2, 2, i)
    df=combined_data[combined_data['Ticker']==company]
    plt.plot(df['date'],df['Adj Close'])
    plt.title(company)

## Time Series of sortlisted stocks for Volume
plt.figure(figsize=(20,12))
for i, company in enumerate(stock_list,1):
    plt.subplot(2, 2, i)
    df=combined_data[combined_data['Ticker']==company]
    plt.plot(df['date'],df['Volume'])
    plt.title(company)

# Time Series of sortlisted stocks for Volume using Ploty dark
for company in (stock_list):
    df=combined_data[combined_data['Ticker']==company]
    fig = px.line(df, x="date", y="Volume", title=company,template = 'plotly_dark')
    fig.show()

# Filter combined data for selected stocks
tesla_data = combined_data[combined_data['Ticker'] == 'TSLA']
apple_data = combined_data[combined_data['Ticker'] == 'AAPL']
amazon_data = combined_data[combined_data['Ticker'] == 'AMZN']
ford_data = combined_data[combined_data['Ticker'] == 'F']
print("Tesla Data:")
print(tesla_data.head())

Tesla Data:
                 Open       High        Low      Close  Adj Close     Volume  \
Date                                                                           
2020-01-02  28.299999  28.713333  28.114000  28.684000  28.684000  142981500   
2020-01-03  29.366667  30.266666  29.128000  29.534000  29.534000  266677500   
2020-01-06  29.364668  30.104000  29.333332  30.102667  30.102667  151995000   
2020-01-07  30.760000  31.441999  30.224001  31.270666  31.270666  268231500   
2020-01-08  31.580000  33.232666  31.215334  32.809334  32.809334  467164500   

           Ticker       date  
Date                          
2020-01-02   TSLA 2020-01-02  
2020-01-03   TSLA 2020-01-03  
2020-01-06   TSLA 2020-01-06  
2020-01-07   TSLA 2020-01-07  
2020-01-08   TSLA 2020-01-08  

close=pd.DataFrame()

close['Tesla']=tesla_data['Close']
close['Apple']=apple_data['Close']
close['Amazon']=amazon_data['Close']
close['Ford']=ford_data['Close']

close.head()

	Tesla	Apple	Amazon	Ford
Date
2020-01-02	28.684000	75.087502	94.900497	9.42
2020-01-03	29.534000	74.357498	93.748497	9.21
2020-01-06	30.102667	74.949997	95.143997	9.16
2020-01-07	31.270666	74.597504	95.343002	9.25
2020-01-08	32.809334	75.797501	94.598503	9.25

#sns.pairplot(data=close)
sns.pairplot(close, kind='reg')

<seaborn.axisgrid.PairGrid at 0x7bd7c1902290>

sns.heatmap(close.corr(),annot=True,cmap='Blues',linecolor="black")

<Axes: >

Above we can see all the relationships on daily returns between all 4 stocks. A quick glance shows an interesting correlation between Ford and Tesla. It might be interesting to investigate that individual comaprison.

# Calculate the 90-day moving average for each stock
moving_avg_90 = close.rolling(window=90).mean()

# Calculate the 180-day moving average for each stock
moving_avg_180 = close.rolling(window=180).mean()

# Display the moving averages
print("90-Day Moving Average:")
print(moving_avg_90.tail())  # Display the last few rows

print("\n180-Day Moving Average:")
print(moving_avg_180.tail())  # Display the last few rows

90-Day Moving Average:
                 Tesla       Apple      Amazon       Ford
Date                                                     
2024-02-23  222.086556  186.390667  149.913666  11.343222
2024-02-26  221.480778  186.417778  150.382333  11.343445
2024-02-27  220.868333  186.478667  150.849777  11.343000
2024-02-28  220.416778  186.540667  151.350111  11.349111
2024-02-29  220.214222  186.599444  151.887444  11.359222

180-Day Moving Average:
                 Tesla       Apple      Amazon       Ford
Date                                                     
2024-02-23  239.191944  184.708000  140.976611  12.228944
2024-02-26  239.052111  184.726556  141.273833  12.219833
2024-02-27  238.856944  184.738000  141.547667  12.211056
2024-02-28  238.621611  184.740556  141.823945  12.203056
2024-02-29  238.355222  184.723667  142.102778  12.195611

# Plotting
plt.figure(figsize=(20, 12))

for stock in close.columns:
    plt.plot(close.index, close[stock], label=f'{stock} Close Price')
    plt.plot(close.index, moving_avg_90[stock], label=f'{stock} 90-Day Moving Average', linestyle='--')
    plt.plot(close.index, moving_avg_180[stock], label=f'{stock} 180-Day Moving Average', linestyle=':')

plt.title('Close Price and Moving Averages for All Stocks')
plt.xlabel('Date')
plt.ylabel('Price')
plt.legend()
plt.xticks(rotation=45)
plt.grid(True)
plt.show()

# Determine the number of subplots needed
num_plots = len(close.columns)
num_rows = 2
num_cols = 2
fig, axs = plt.subplots(num_rows, num_cols, figsize=(25, 12))
axs = axs.flatten()
for i, stock in enumerate(close.columns):
    axs[i].plot(close.index, close[stock], label='Close Price', color='blue')
    axs[i].plot(close.index, moving_avg_90[stock], label='90-Day Moving Average', color='red')
    axs[i].plot(close.index, moving_avg_180[stock], label='180-Day Moving Average', color='green')
    axs[i].set_title(f'{stock} Close Price and Moving Averages')
    axs[i].set_xlabel('Date')
    axs[i].set_ylabel('Price')
    axs[i].legend()
    axs[i].tick_params(axis='x', rotation=45)
    axs[i].grid(True)
plt.tight_layout()
plt.show()

Time Series Decomposition

# Perform time series decomposition for each stock
decomposition = {}
for stock in close.columns:
    result = seasonal_decompose(close[stock], model='additive', period=365)
    decomposition[stock] = result
for stock, result in decomposition.items():
    plt.figure(figsize=(20, 20))
    plt.subplot(411)
    plt.plot(result.observed, label='Original', color='blue')
    plt.legend(loc='upper left')
    plt.subplot(412)
    plt.plot(result.trend, label='Trend', color='red')
    plt.legend(loc='upper left')
    plt.subplot(413)
    plt.plot(result.seasonal, label='Seasonal', color='green')
    plt.legend(loc='upper left')
    plt.subplot(414)
    plt.plot(result.resid, label='Residual', color='purple')
    plt.legend(loc='upper left')
    plt.suptitle(f'Decomposition of {stock} Close Price')
    plt.tight_layout()
    plt.show()

Description of Time series

# Iterate over each stock and perform the ADF test
for stock in close.columns:
    ts = close[stock]
    result = adfuller(ts)
    adf_statistic = result[0]
    p_value = result[1]
    critical_values = result[4]
    print(f'Stock: {stock}')
    print(f'ADF Statistic: {adf_statistic}')
    print(f'p-value: {p_value}')
    print('Critical Values:')
    for key, value in critical_values.items():
        print(f'   {key}: {value}')
    if p_value < 0.05:
        print('Result: Series is likely stationary (reject null hypothesis)')
    else:
        print('Result: Series is likely non-stationary (fail to reject null hypothesis)')
    print()

Stock: Tesla
ADF Statistic: -2.508930972626126
p-value: 0.1133164174563997
Critical Values:
   1%: -3.436671659540904
   5%: -2.8643311477003515
   10%: -2.5682561222519897
Result: Series is likely non-stationary (fail to reject null hypothesis)

Stock: Apple
ADF Statistic: -1.7984539561652384
p-value: 0.38122953117045855
Critical Values:
   1%: -3.436617132314828
   5%: -2.864307097893787
   10%: -2.568243313067353
Result: Series is likely non-stationary (fail to reject null hypothesis)

Stock: Amazon
ADF Statistic: -1.8994410397299941
p-value: 0.3323241089508116
Critical Values:
   1%: -3.436617132314828
   5%: -2.864307097893787
   10%: -2.568243313067353
Result: Series is likely non-stationary (fail to reject null hypothesis)

Stock: Ford
ADF Statistic: -1.9927383413967492
p-value: 0.28976613221893677
Critical Values:
   1%: -3.4367333690404767
   5%: -2.8643583648001925
   10%: -2.568270618452702
Result: Series is likely non-stationary (fail to reject null hypothesis)

# Iterate over each stock and check for trend and seasonality using autocorrelation
for stock in close.columns:
    ts = close[stock]
    autocorr = acf(ts, fft=False)
    trend_detected = autocorr[1] > 0.5
    seasonal_detected = any(autocorr[lag] > 0.5 for lag in range(7, 31))
    print(f'Stock: {stock}')
    if trend_detected:
        print('Trend detected')
    else:
        print('No trend detected')
    if seasonal_detected:
        print('Seasonality detected')
    else:
        print('No seasonality detected')
    print()

Stock: Tesla
Trend detected
Seasonality detected

Stock: Apple
Trend detected
Seasonality detected

Stock: Amazon
Trend detected
Seasonality detected

Stock: Ford
Trend detected
Seasonality detected

TS models

# Function to perform the ADF test and check stationarity
def check_stationarity(data, stock_name, transformation_name):
    result = adfuller(data)
    p_value = result[1]
    if p_value < 0.05:
        print(f"{stock_name} ({transformation_name}): ADF Statistic:", result[0])
        print(f"{stock_name} ({transformation_name}): p-value:", p_value)
        print(f"{stock_name} ({transformation_name}): Reject the null hypothesis (data is stationary)")
    else:
        print(f"{stock_name} ({transformation_name}): ADF Statistic:", result[0])
        print(f"{stock_name} ({transformation_name}): p-value:", p_value)
        print(f"{stock_name} ({transformation_name}): Fail to reject the null hypothesis (data is non-stationary)")

for stock_name in close.columns:
    print("\nStock:", stock_name)
    ts = close[stock_name]
    print("Original Data:")
    check_stationarity(ts, stock_name, "Original")

    # First-order differencing
    print("\nFirst-order Differencing:")
    ts_diff = ts.diff().dropna()
    check_stationarity(ts_diff, stock_name, "First-order Differencing")

    # Logarithmic transformation
    print("\nLogarithmic Transformation:")
    ts_log = np.log(ts)
    check_stationarity(ts_log, stock_name, "Logarithmic Transformation")

    # Box-Cox Transformation
    print("\nBox-Cox Transformation:")
    ts_boxcox, _ = boxcox(ts)
    check_stationarity(ts_boxcox, stock_name, "Box-Cox Transformation")

Stock: Tesla
Original Data:
Tesla (Original): ADF Statistic: -2.508930972626126
Tesla (Original): p-value: 0.1133164174563997
Tesla (Original): Fail to reject the null hypothesis (data is non-stationary)

First-order Differencing:
Tesla (First-order Differencing): ADF Statistic: -9.332542763008549
Tesla (First-order Differencing): p-value: 9.245353038076098e-16
Tesla (First-order Differencing): Reject the null hypothesis (data is stationary)

Logarithmic Transformation:
Tesla (Logarithmic Transformation): ADF Statistic: -3.319207332842492
Tesla (Logarithmic Transformation): p-value: 0.01404143513690903
Tesla (Logarithmic Transformation): Reject the null hypothesis (data is stationary)

Box-Cox Transformation:
Tesla (Box-Cox Transformation): ADF Statistic: -2.5143436530272614
Tesla (Box-Cox Transformation): p-value: 0.11202325085143111
Tesla (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary)

Stock: Apple
Original Data:
Apple (Original): ADF Statistic: -1.7984539561652384
Apple (Original): p-value: 0.38122953117045855
Apple (Original): Fail to reject the null hypothesis (data is non-stationary)

First-order Differencing:
Apple (First-order Differencing): ADF Statistic: -33.39772071161903
Apple (First-order Differencing): p-value: 0.0
Apple (First-order Differencing): Reject the null hypothesis (data is stationary)

Logarithmic Transformation:
Apple (Logarithmic Transformation): ADF Statistic: -1.914122732950772
Apple (Logarithmic Transformation): p-value: 0.32544677573603575
Apple (Logarithmic Transformation): Fail to reject the null hypothesis (data is non-stationary)

Box-Cox Transformation:
Apple (Box-Cox Transformation): ADF Statistic: -1.6634591706048383
Apple (Box-Cox Transformation): p-value: 0.4500308740323451
Apple (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary)

Stock: Amazon
Original Data:
Amazon (Original): ADF Statistic: -1.8994410397299941
Amazon (Original): p-value: 0.3323241089508116
Amazon (Original): Fail to reject the null hypothesis (data is non-stationary)

First-order Differencing:
Amazon (First-order Differencing): ADF Statistic: -32.991375934978585
Amazon (First-order Differencing): p-value: 0.0
Amazon (First-order Differencing): Reject the null hypothesis (data is stationary)

Logarithmic Transformation:
Amazon (Logarithmic Transformation): ADF Statistic: -1.9734711451741194
Amazon (Logarithmic Transformation): p-value: 0.2983248224454217
Amazon (Logarithmic Transformation): Fail to reject the null hypothesis (data is non-stationary)

Box-Cox Transformation:
Amazon (Box-Cox Transformation): ADF Statistic: -1.891093228427692
Amazon (Box-Cox Transformation): p-value: 0.336262668290978
Amazon (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary)

Stock: Ford
Original Data:
Ford (Original): ADF Statistic: -1.9927383413967492
Ford (Original): p-value: 0.28976613221893677
Ford (Original): Fail to reject the null hypothesis (data is non-stationary)

First-order Differencing:
Ford (First-order Differencing): ADF Statistic: -31.107163119871892
Ford (First-order Differencing): p-value: 0.0
Ford (First-order Differencing): Reject the null hypothesis (data is stationary)

Logarithmic Transformation:
Ford (Logarithmic Transformation): ADF Statistic: -1.6174802251765108
Ford (Logarithmic Transformation): p-value: 0.47405056419435393
Ford (Logarithmic Transformation): Fail to reject the null hypothesis (data is non-stationary)

Box-Cox Transformation:
Ford (Box-Cox Transformation): ADF Statistic: -1.522474746257365
Ford (Box-Cox Transformation): p-value: 0.5223655614334552
Ford (Box-Cox Transformation): Fail to reject the null hypothesis (data is non-stationary)

# Function to visualize the data
def plot_data(data, title):
    plt.figure(figsize=(10, 6))
    plt.plot(data.index, data.values, label='Original Data')
    plt.xlabel('Date')
    plt.ylabel('Value')
    plt.title(title)
    plt.legend()
    plt.show()

# Function to perform ACF and PACF analysis
def acf_pacf_analysis(data, title):
    plt.figure(figsize=(12, 6))
    plot_acf(data, lags=20)
    plt.title(f'Autocorrelation Function (ACF) - {title}')
    plt.show()

    plt.figure(figsize=(12, 6))
    plot_pacf(data, lags=20)
    plt.title(f'Partial Autocorrelation Function (PACF) - {title}')
    plt.show()

# Function for ARIMA model
def fit_arima(data, order):
    model_arima = ARIMA(data, order=order)
    model_fit_arima = model_arima.fit()
    return model_fit_arima

# Function for SARIMA model
def fit_sarima(data, order, seasonal_order):
    model_sarima = SARIMAX(data, order=order, seasonal_order=seasonal_order)
    model_fit_sarima = model_sarima.fit()
    return model_fit_sarima

# Function for Exponential Smoothing model
def fit_exponential_smoothing(data):
    model_es = ExponentialSmoothing(data)
    model_fit_es = model_es.fit()
    return model_fit_es

# Iterate over each stock and perform model selection
for stock_name in close.columns:
    print("\nStock:", stock_name)
    ts_diff = close[stock_name].diff().dropna()
    plot_data(ts_diff, f'Transformed Data - {stock_name}')

    acf_pacf_analysis(ts_diff, stock_name)

    # Fit ARIMA model
    model_fit_arima = fit_arima(ts_diff, order=(1, 1, 1))
    print("ARIMA Model Summary:")
    print(model_fit_arima.summary())

    # Fit SARIMA model
    model_fit_sarima = fit_sarima(ts_diff, order=(1, 1, 1), seasonal_order=(1, 1, 1, 12))
    print("SARIMA Model Summary:")
    print(model_fit_sarima.summary())

    # Fit Exponential Smoothing model
    model_fit_es = fit_exponential_smoothing(ts_diff)
    print("Exponential Smoothing Model Summary:")
    print(model_fit_es.summary())

Stock: Tesla

<Figure size 1200x600 with 0 Axes>

<Figure size 1200x600 with 0 Axes>

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

ARIMA Model Summary:
                               SARIMAX Results                                
==============================================================================
Dep. Variable:                  Tesla   No. Observations:                 1046
Model:                 ARIMA(1, 1, 1)   Log Likelihood               -3717.950
Date:                Fri, 29 Mar 2024   AIC                           7441.900
Time:                        12:17:45   BIC                           7456.755
Sample:                             0   HQIC                          7447.534
                               - 1046                                         
Covariance Type:                  opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1         -0.0287      0.027     -1.064      0.287      -0.082       0.024
ma.L1         -0.9986      0.005   -186.835      0.000      -1.009      -0.988
sigma2        71.6794      1.858     38.588      0.000      68.039      75.320
===================================================================================
Ljung-Box (L1) (Q):                   0.00   Jarque-Bera (JB):               633.80
Prob(Q):                              0.96   Prob(JB):                         0.00
Heteroskedasticity (H):               1.09   Skew:                            -0.18
Prob(H) (two-sided):                  0.43   Kurtosis:                         6.80
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

SARIMA Model Summary:
                                     SARIMAX Results                                      
==========================================================================================
Dep. Variable:                              Tesla   No. Observations:                 1046
Model:             SARIMAX(1, 1, 1)x(1, 1, 1, 12)   Log Likelihood               -3705.625
Date:                            Fri, 29 Mar 2024   AIC                           7421.250
Time:                                    12:17:59   BIC                           7445.951
Sample:                                         0   HQIC                          7430.623
                                           - 1046                                         
Covariance Type:                              opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1         -0.0269      0.028     -0.970      0.332      -0.081       0.027
ma.L1         -0.9999      0.339     -2.952      0.003      -1.664      -0.336
ar.S.L12       0.0223      0.032      0.690      0.490      -0.041       0.086
ma.S.L12      -0.9999      2.487     -0.402      0.688      -5.874       3.874
sigma2        71.6939    179.443      0.400      0.689    -280.008     423.395
===================================================================================
Ljung-Box (L1) (Q):                   0.00   Jarque-Bera (JB):               603.44
Prob(Q):                              0.99   Prob(JB):                         0.00
Heteroskedasticity (H):               1.09   Skew:                            -0.26
Prob(H) (two-sided):                  0.43   Kurtosis:                         6.71
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
                       ExponentialSmoothing Model Results                       
================================================================================
Dep. Variable:                    Tesla   No. Observations:                 1046
Model:             ExponentialSmoothing   SSE                          75124.211
Optimized:                         True   AIC                           4474.781
Trend:                             None   BIC                           4484.687
Seasonal:                          None   AICC                          4474.820
Seasonal Periods:                  None   Date:                 Fri, 29 Mar 2024
Box-Cox:                          False   Time:                         12:17:59
Box-Cox Coeff.:                    None                                         
==============================================================================
                       coeff                 code              optimized      
------------------------------------------------------------------------------
smoothing_level            0.0050151                alpha                 True
initial_level              0.5013415                  l.0                 True
------------------------------------------------------------------------------

Stock: Apple

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

<Figure size 1200x600 with 0 Axes>

<Figure size 1200x600 with 0 Axes>

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

ARIMA Model Summary:
                               SARIMAX Results                                
==============================================================================
Dep. Variable:                  Apple   No. Observations:                 1046
Model:                 ARIMA(1, 1, 1)   Log Likelihood               -2497.041
Date:                Fri, 29 Mar 2024   AIC                           5000.081
Time:                        12:18:01   BIC                           5014.937
Sample:                             0   HQIC                          5005.715
                               - 1046                                         
Covariance Type:                  opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1         -0.0326      0.027     -1.226      0.220      -0.085       0.019
ma.L1         -1.0000      0.252     -3.972      0.000      -1.493      -0.507
sigma2         6.9202      1.752      3.950      0.000       3.486      10.354
===================================================================================
Ljung-Box (L1) (Q):                   0.00   Jarque-Bera (JB):               120.09
Prob(Q):                              0.96   Prob(JB):                         0.00
Heteroskedasticity (H):               0.93   Skew:                            -0.06
Prob(H) (two-sided):                  0.53   Kurtosis:                         4.66
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

SARIMA Model Summary:
                                     SARIMAX Results                                      
==========================================================================================
Dep. Variable:                              Apple   No. Observations:                 1046
Model:             SARIMAX(1, 1, 1)x(1, 1, 1, 12)   Log Likelihood               -2499.677
Date:                            Fri, 29 Mar 2024   AIC                           5009.353
Time:                                    12:18:12   BIC                           5034.054
Sample:                                         0   HQIC                          5018.726
                                           - 1046                                         
Covariance Type:                              opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1         -0.0314      0.028     -1.140      0.254      -0.085       0.023
ma.L1         -0.9996      0.077    -13.021      0.000      -1.150      -0.849
ar.S.L12      -0.0259      0.031     -0.844      0.399      -0.086       0.034
ma.S.L12      -0.9995      0.625     -1.598      0.110      -2.225       0.226
sigma2         6.9389      4.357      1.593      0.111      -1.600      15.478
===================================================================================
Ljung-Box (L1) (Q):                   0.00   Jarque-Bera (JB):               106.67
Prob(Q):                              0.96   Prob(JB):                         0.00
Heteroskedasticity (H):               0.94   Skew:                            -0.02
Prob(H) (two-sided):                  0.57   Kurtosis:                         4.57
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
                       ExponentialSmoothing Model Results                       
================================================================================
Dep. Variable:                    Apple   No. Observations:                 1046
Model:             ExponentialSmoothing   SSE                           7239.659
Optimized:                         True   AIC                           2027.592
Trend:                             None   BIC                           2037.498
Seasonal:                          None   AICC                          2027.631
Seasonal Periods:                  None   Date:                 Fri, 29 Mar 2024
Box-Cox:                          False   Time:                         12:18:12
Box-Cox Coeff.:                    None                                         
==============================================================================
                       coeff                 code              optimized      
------------------------------------------------------------------------------
smoothing_level           1.4901e-08                alpha                 True
initial_level              0.1010158                  l.0                 True
------------------------------------------------------------------------------

Stock: Amazon

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

<Figure size 1200x600 with 0 Axes>

<Figure size 1200x600 with 0 Axes>

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

ARIMA Model Summary:
                               SARIMAX Results                                
==============================================================================
Dep. Variable:                 Amazon   No. Observations:                 1046
Model:                 ARIMA(1, 1, 1)   Log Likelihood               -2667.438
Date:                Fri, 29 Mar 2024   AIC                           5340.875
Time:                        12:18:14   BIC                           5355.730
Sample:                             0   HQIC                          5346.509
                               - 1046                                         
Covariance Type:                  opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1         -0.0209      0.026     -0.797      0.426      -0.072       0.031
ma.L1         -0.9999      0.051    -19.718      0.000      -1.099      -0.901
sigma2         9.5893      0.520     18.435      0.000       8.570      10.609
===================================================================================
Ljung-Box (L1) (Q):                   0.00   Jarque-Bera (JB):               707.24
Prob(Q):                              0.97   Prob(JB):                         0.00
Heteroskedasticity (H):               0.77   Skew:                            -0.07
Prob(H) (two-sided):                  0.01   Kurtosis:                         7.03
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

SARIMA Model Summary:
                                     SARIMAX Results                                      
==========================================================================================
Dep. Variable:                             Amazon   No. Observations:                 1046
Model:             SARIMAX(1, 1, 1)x(1, 1, 1, 12)   Log Likelihood               -2667.867
Date:                            Fri, 29 Mar 2024   AIC                           5345.733
Time:                                    12:18:23   BIC                           5370.434
Sample:                                         0   HQIC                          5355.107
                                           - 1046                                         
Covariance Type:                              opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1         -0.0213      0.028     -0.759      0.448      -0.076       0.034
ma.L1         -0.9938      0.007   -141.399      0.000      -1.008      -0.980
ar.S.L12      -0.0180      0.032     -0.570      0.568      -0.080       0.044
ma.S.L12      -0.9946      0.065    -15.187      0.000      -1.123      -0.866
sigma2         9.6900      0.601     16.115      0.000       8.512      10.869
===================================================================================
Ljung-Box (L1) (Q):                   0.00   Jarque-Bera (JB):               661.93
Prob(Q):                              0.96   Prob(JB):                         0.00
Heteroskedasticity (H):               0.79   Skew:                             0.01
Prob(H) (two-sided):                  0.03   Kurtosis:                         6.92
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
                       ExponentialSmoothing Model Results                       
================================================================================
Dep. Variable:                   Amazon   No. Observations:                 1046
Model:             ExponentialSmoothing   SSE                          10024.847
Optimized:                         True   AIC                           2368.058
Trend:                             None   BIC                           2377.963
Seasonal:                          None   AICC                          2368.096
Seasonal Periods:                  None   Date:                 Fri, 29 Mar 2024
Box-Cox:                          False   Time:                         12:18:23
Box-Cox Coeff.:                    None                                         
==============================================================================
                       coeff                 code              optimized      
------------------------------------------------------------------------------
smoothing_level           1.4901e-08                alpha                 True
initial_level              0.0782595                  l.0                 True
------------------------------------------------------------------------------

Stock: Ford

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

<Figure size 1200x600 with 0 Axes>

<Figure size 1200x600 with 0 Axes>

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

ARIMA Model Summary:
                               SARIMAX Results                                
==============================================================================
Dep. Variable:                   Ford   No. Observations:                 1046
Model:                 ARIMA(1, 1, 1)   Log Likelihood                -396.431
Date:                Fri, 29 Mar 2024   AIC                            798.863
Time:                        12:18:26   BIC                            813.718
Sample:                             0   HQIC                           804.497
                               - 1046                                         
Covariance Type:                  opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1          0.0385      0.021      1.813      0.070      -0.003       0.080
ma.L1         -0.9995      0.015    -64.614      0.000      -1.030      -0.969
sigma2         0.1243      0.003     40.394      0.000       0.118       0.130
===================================================================================
Ljung-Box (L1) (Q):                   0.00   Jarque-Bera (JB):              1580.57
Prob(Q):                              0.99   Prob(JB):                         0.00
Heteroskedasticity (H):               1.24   Skew:                             0.03
Prob(H) (two-sided):                  0.05   Kurtosis:                         9.02
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

SARIMA Model Summary:
                                     SARIMAX Results                                      
==========================================================================================
Dep. Variable:                               Ford   No. Observations:                 1046
Model:             SARIMAX(1, 1, 1)x(1, 1, 1, 12)   Log Likelihood                -422.987
Date:                            Fri, 29 Mar 2024   AIC                            855.973
Time:                                    12:18:39   BIC                            880.675
Sample:                                         0   HQIC                           865.347
                                           - 1046                                         
Covariance Type:                              opg                                         
==============================================================================
                 coef    std err          z      P>|z|      [0.025      0.975]
------------------------------------------------------------------------------
ar.L1          0.0355      0.022      1.643      0.100      -0.007       0.078
ma.L1         -0.9979      0.012    -85.196      0.000      -1.021      -0.975
ar.S.L12       0.0183      0.024      0.760      0.447      -0.029       0.065
ma.S.L12      -0.9993      0.501     -1.993      0.046      -1.982      -0.017
sigma2         0.1248      0.062      2.025      0.043       0.004       0.246
===================================================================================
Ljung-Box (L1) (Q):                   0.01   Jarque-Bera (JB):              1398.64
Prob(Q):                              0.94   Prob(JB):                         0.00
Heteroskedasticity (H):               1.16   Skew:                             0.07
Prob(H) (two-sided):                  0.18   Kurtosis:                         8.70
===================================================================================

Warnings:
[1] Covariance matrix calculated using the outer product of gradients (complex-step).
Exponential Smoothing Model Summary:
                       ExponentialSmoothing Model Results                       
================================================================================
Dep. Variable:                     Ford   No. Observations:                 1046
Model:             ExponentialSmoothing   SSE                            129.987
Optimized:                         True   AIC                          -2177.220
Trend:                             None   BIC                          -2167.315
Seasonal:                          None   AICC                         -2177.182
Seasonal Periods:                  None   Date:                 Fri, 29 Mar 2024
Box-Cox:                          False   Time:                         12:18:39
Box-Cox Coeff.:                    None                                         
==============================================================================
                       coeff                 code              optimized      
------------------------------------------------------------------------------
smoothing_level           1.4901e-08                alpha                 True
initial_level              0.0028872                  l.0                 True
------------------------------------------------------------------------------

/usr/local/lib/python3.10/dist-packages/statsmodels/tsa/base/tsa_model.py:473: ValueWarning:

A date index has been provided, but it has no associated frequency information and so will be ignored when e.g. forecasting.

Tesla

ARIMA Model:

AIC: 7441.900 BIC: 7456.755 Log Likelihood: -3717.950 Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 633.80 SARIMA Model:

AIC: 7421.250 BIC: 7445.951 Log Likelihood: -3705.625 Ljung-Box (L1) (Q): 0.00 Jarque-Bera (JB): 603.44 Exponential Smoothing Model:

SSE: 75124.211 AIC: 4474.781 BIC: 4484.687

Apple

ARIMA Model:

AIC: 5000.081 BIC: 5014.937 Log Likelihood: -2497.041 SARIMA Model:

AIC: 5009.353 BIC: 5034.054 Log Likelihood: -2499.677 Exponential Smoothing Model:

AIC: 2027.592 BIC: 2037.498 SSE: 7239.659

Amazon

ARIMA Model:

AIC: 5340.875 BIC: 5355.730 Log Likelihood: -2667.438 SARIMA Model:

AIC: 5345.733 BIC: 5370.434 Log Likelihood: -2667.867 Exponential Smoothing Model:

AIC: 2368.058 BIC: 2377.963 SSE: 10024.847

Ford

ARIMA Model:

AIC: 798.863 BIC: 813.718 Log Likelihood: -396.431 SARIMA Model:

AIC: 855.973 BIC: 880.675 Log Likelihood: -422.987 Exponential Smoothing Model:

AIC: -2177.220 BIC: -2167.315 SSE: 129.987

# Perform model selection
for stock_name in ['Tesla', 'Apple', 'Amazon', 'Ford']:
    print("\nStock:", stock_name)
    ts_diff = close[stock_name].diff().dropna()
    aic_values = []
    bic_values = []

    # ARIMA Model
    aic_values.append(798.863)
    bic_values.append(813.718)

    # SARIMA Model
    aic_values.append(855.973)
    bic_values.append(880.675)

    # Exponential Smoothing Model
    aic_values.append(-2177.220)
    bic_values.append(-2167.315)

    print(aic_values)
    print(bic_values)

    # Model Selection
    best_model = None
    min_aic = min(aic_values)
    min_bic = min(bic_values)
    if min_aic == min_bic:
        best_model = "Exponential Smoothing"
    elif min_aic < min_bic:
        best_model = "ARIMA"
    else:
        best_model = "SARIMA"

    print("Best Model:", best_model)

Stock: Tesla
[798.863, 855.973, -2177.22]
[813.718, 880.675, -2167.315]
Best Model: ARIMA

Stock: Apple
[798.863, 855.973, -2177.22]
[813.718, 880.675, -2167.315]
Best Model: ARIMA

Stock: Amazon
[798.863, 855.973, -2177.22]
[813.718, 880.675, -2167.315]
Best Model: ARIMA

Stock: Ford
[798.863, 855.973, -2177.22]
[813.718, 880.675, -2167.315]
Best Model: ARIMA

Lets do the model and test with one stock Apple

# Split the data into a training and validation dataset

# Load the dataset
series = read_csv('/content/drive/MyDrive/data/tmsp2/AAPL_historical_data.csv', header=0, index_col=0, parse_dates=True, squeeze=True)
series = series['Close']
# Define the split point
split_point = len(series) - 100

# Split the dataset
dataset, validation = series[0:split_point], series[split_point:]

# Print the sizes of the dataset and validation sets
print('Dataset %d, Validation %d' % (len(dataset), len(validation)))

# Save the datasets to CSV files
dataset.to_csv('/content/drive/MyDrive/data/app_dataset.csv', header=False)
validation.to_csv('/content/drive/MyDrive/data/app_validation.csv', header=False)

<ipython-input-4-79f3e07e30f5>:5: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/tmsp2/AAPL_historical_data.csv', header=0, index_col=0, parse_dates=True, squeeze=True)

Dataset 947, Validation 100

# Read the dataset and validation CSV files
dataset = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None)
validation = read_csv('/content/drive/MyDrive/data/app_validation.csv', header=None)

# Print the number of observations in each file
print('Number of observations in dataset:', len(dataset))
print('Number of observations in validation:', len(validation))

Number of observations in dataset: 947
Number of observations in validation: 100

dataset

	0	1
0	2020-01-02	75.087502
1	2020-01-03	74.357498
2	2020-01-06	74.949997
3	2020-01-07	74.597504
4	2020-01-08	75.797501
...	...	...
942	2023-09-29	171.210007
943	2023-10-02	173.750000
944	2023-10-03	172.399994
945	2023-10-04	173.660004
946	2023-10-05	174.910004

947 rows × 2 columns

# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True) # prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]

# walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):

# predict
 yhat = history[-1]
 predictions.append(yhat)

# observation
 obs = test[i]
 history.append(obs)
 print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))

# report performance
rmse = sqrt(mean_squared_error(test, predictions))
print('RMSE: %.3f' % rmse)

>Predicted=150.000, Expected=151.000
>Predicted=151.000, Expected=153.490
>Predicted=153.490, Expected=157.870
>Predicted=157.870, Expected=160.550
>Predicted=160.550, Expected=161.020
>Predicted=161.020, Expected=161.410
>Predicted=161.410, Expected=161.940
>Predicted=161.940, Expected=156.810
>Predicted=156.810, Expected=160.240
>Predicted=160.240, Expected=165.300
>Predicted=165.300, Expected=164.770
>Predicted=164.770, Expected=163.760
>Predicted=163.760, Expected=161.840
>Predicted=161.840, Expected=165.320
>Predicted=165.320, Expected=171.180
>Predicted=171.180, Expected=175.080
>Predicted=175.080, Expected=174.560
>Predicted=174.560, Expected=179.450
>Predicted=179.450, Expected=175.740
>Predicted=175.740, Expected=174.330
>Predicted=174.330, Expected=179.300
>Predicted=179.300, Expected=172.260
>Predicted=172.260, Expected=171.140
>Predicted=171.140, Expected=169.750
>Predicted=169.750, Expected=172.990
>Predicted=172.990, Expected=175.640
>Predicted=175.640, Expected=176.280
>Predicted=176.280, Expected=180.330
>Predicted=180.330, Expected=179.290
>Predicted=179.290, Expected=179.380
>Predicted=179.380, Expected=178.200
>Predicted=178.200, Expected=177.570
>Predicted=177.570, Expected=182.010
>Predicted=182.010, Expected=179.700
>Predicted=179.700, Expected=174.920
>Predicted=174.920, Expected=172.000
>Predicted=172.000, Expected=172.170
>Predicted=172.170, Expected=172.190
>Predicted=172.190, Expected=175.080
>Predicted=175.080, Expected=175.530
>Predicted=175.530, Expected=172.190
>Predicted=172.190, Expected=173.070
>Predicted=173.070, Expected=169.800
>Predicted=169.800, Expected=166.230
>Predicted=166.230, Expected=164.510
>Predicted=164.510, Expected=162.410
>Predicted=162.410, Expected=161.620
>Predicted=161.620, Expected=159.780
>Predicted=159.780, Expected=159.690
>Predicted=159.690, Expected=159.220
>Predicted=159.220, Expected=170.330
>Predicted=170.330, Expected=174.780
>Predicted=174.780, Expected=174.610
>Predicted=174.610, Expected=175.840
>Predicted=175.840, Expected=172.900
>Predicted=172.900, Expected=172.390
>Predicted=172.390, Expected=171.660
>Predicted=171.660, Expected=174.830
>Predicted=174.830, Expected=176.280
>Predicted=176.280, Expected=172.120
>Predicted=172.120, Expected=168.640
>Predicted=168.640, Expected=168.880
>Predicted=168.880, Expected=172.790
>Predicted=172.790, Expected=172.550
>Predicted=172.550, Expected=168.880
>Predicted=168.880, Expected=167.300
>Predicted=167.300, Expected=164.320
>Predicted=164.320, Expected=160.070
>Predicted=160.070, Expected=162.740
>Predicted=162.740, Expected=164.850
>Predicted=164.850, Expected=165.120
>Predicted=165.120, Expected=163.200
>Predicted=163.200, Expected=166.560
>Predicted=166.560, Expected=166.230
>Predicted=166.230, Expected=163.170
>Predicted=163.170, Expected=159.300
>Predicted=159.300, Expected=157.440
>Predicted=157.440, Expected=162.950
>Predicted=162.950, Expected=158.520
>Predicted=158.520, Expected=154.730
>Predicted=154.730, Expected=150.620
>Predicted=150.620, Expected=155.090
>Predicted=155.090, Expected=159.590
>Predicted=159.590, Expected=160.620
>Predicted=160.620, Expected=163.980
>Predicted=163.980, Expected=165.380
>Predicted=165.380, Expected=168.820
>Predicted=168.820, Expected=170.210
>Predicted=170.210, Expected=174.070
>Predicted=174.070, Expected=174.720
>Predicted=174.720, Expected=175.600
>Predicted=175.600, Expected=178.960
>Predicted=178.960, Expected=177.770
>Predicted=177.770, Expected=174.610
>Predicted=174.610, Expected=174.310
>Predicted=174.310, Expected=178.440
>Predicted=178.440, Expected=175.060
>Predicted=175.060, Expected=171.830
>Predicted=171.830, Expected=172.140
>Predicted=172.140, Expected=170.090
>Predicted=170.090, Expected=165.750
>Predicted=165.750, Expected=167.660
>Predicted=167.660, Expected=170.400
>Predicted=170.400, Expected=165.290
>Predicted=165.290, Expected=165.070
>Predicted=165.070, Expected=167.400
>Predicted=167.400, Expected=167.230
>Predicted=167.230, Expected=166.420
>Predicted=166.420, Expected=161.790
>Predicted=161.790, Expected=162.880
>Predicted=162.880, Expected=156.800
>Predicted=156.800, Expected=156.570
>Predicted=156.570, Expected=163.640
>Predicted=163.640, Expected=157.650
>Predicted=157.650, Expected=157.960
>Predicted=157.960, Expected=159.480
>Predicted=159.480, Expected=166.020
>Predicted=166.020, Expected=156.770
>Predicted=156.770, Expected=157.280
>Predicted=157.280, Expected=152.060
>Predicted=152.060, Expected=154.510
>Predicted=154.510, Expected=146.500
>Predicted=146.500, Expected=142.560
>Predicted=142.560, Expected=147.110
>Predicted=147.110, Expected=145.540
>Predicted=145.540, Expected=149.240
>Predicted=149.240, Expected=140.820
>Predicted=140.820, Expected=137.350
>Predicted=137.350, Expected=137.590
>Predicted=137.590, Expected=143.110
>Predicted=143.110, Expected=140.360
>Predicted=140.360, Expected=140.520
>Predicted=140.520, Expected=143.780
>Predicted=143.780, Expected=149.640
>Predicted=149.640, Expected=148.840
>Predicted=148.840, Expected=148.710
>Predicted=148.710, Expected=151.210
>Predicted=151.210, Expected=145.380
>Predicted=145.380, Expected=146.140
>Predicted=146.140, Expected=148.710
>Predicted=148.710, Expected=147.960
>Predicted=147.960, Expected=142.640
>Predicted=142.640, Expected=137.130
>Predicted=137.130, Expected=131.880
>Predicted=131.880, Expected=132.760
>Predicted=132.760, Expected=135.430
>Predicted=135.430, Expected=130.060
>Predicted=130.060, Expected=131.560
>Predicted=131.560, Expected=135.870
>Predicted=135.870, Expected=135.350
>Predicted=135.350, Expected=138.270
>Predicted=138.270, Expected=141.660
>Predicted=141.660, Expected=141.660
>Predicted=141.660, Expected=137.440
>Predicted=137.440, Expected=139.230
>Predicted=139.230, Expected=136.720
>Predicted=136.720, Expected=138.930
>Predicted=138.930, Expected=141.560
>Predicted=141.560, Expected=142.920
>Predicted=142.920, Expected=146.350
>Predicted=146.350, Expected=147.040
>Predicted=147.040, Expected=144.870
>Predicted=144.870, Expected=145.860
>Predicted=145.860, Expected=145.490
>Predicted=145.490, Expected=148.470
>Predicted=148.470, Expected=150.170
>Predicted=150.170, Expected=147.070
>Predicted=147.070, Expected=151.000
>Predicted=151.000, Expected=153.040
>Predicted=153.040, Expected=155.350
>Predicted=155.350, Expected=154.090
>Predicted=154.090, Expected=152.950
>Predicted=152.950, Expected=151.600
>Predicted=151.600, Expected=156.790
>Predicted=156.790, Expected=157.350
>Predicted=157.350, Expected=162.510
>Predicted=162.510, Expected=161.510
>Predicted=161.510, Expected=160.010
>Predicted=160.010, Expected=166.130
>Predicted=166.130, Expected=165.810
>Predicted=165.810, Expected=165.350
>Predicted=165.350, Expected=164.870
>Predicted=164.870, Expected=164.920
>Predicted=164.920, Expected=169.240
>Predicted=169.240, Expected=168.490
>Predicted=168.490, Expected=172.100
>Predicted=172.100, Expected=173.190
>Predicted=173.190, Expected=173.030
>Predicted=173.030, Expected=174.550
>Predicted=174.550, Expected=174.150
>Predicted=174.150, Expected=171.520
>Predicted=171.520, Expected=167.570
>Predicted=167.570, Expected=167.230
>Predicted=167.230, Expected=167.530
>Predicted=167.530, Expected=170.030
>Predicted=170.030, Expected=163.620
>Predicted=163.620, Expected=161.380
>Predicted=161.380, Expected=158.910
>Predicted=158.910, Expected=157.220
>Predicted=157.220, Expected=157.960
>Predicted=157.960, Expected=155.810
>Predicted=155.810, Expected=154.530
>Predicted=154.530, Expected=155.960
>Predicted=155.960, Expected=154.460
>Predicted=154.460, Expected=157.370
>Predicted=157.370, Expected=163.430
>Predicted=163.430, Expected=153.840
>Predicted=153.840, Expected=155.310
>Predicted=155.310, Expected=152.370
>Predicted=152.370, Expected=150.700
>Predicted=150.700, Expected=154.480
>Predicted=154.480, Expected=156.900
>Predicted=156.900, Expected=153.720
>Predicted=153.720, Expected=152.740
>Predicted=152.740, Expected=150.430
>Predicted=150.430, Expected=150.770
>Predicted=150.770, Expected=151.760
>Predicted=151.760, Expected=149.840
>Predicted=149.840, Expected=142.480
>Predicted=142.480, Expected=138.200
>Predicted=138.200, Expected=142.450
>Predicted=142.450, Expected=146.100
>Predicted=146.100, Expected=146.400
>Predicted=146.400, Expected=145.430
>Predicted=145.430, Expected=140.090
>Predicted=140.090, Expected=140.420
>Predicted=140.420, Expected=138.980
>Predicted=138.980, Expected=138.340
>Predicted=138.340, Expected=142.990
>Predicted=142.990, Expected=138.380
>Predicted=138.380, Expected=142.410
>Predicted=142.410, Expected=143.750
>Predicted=143.750, Expected=143.860
>Predicted=143.860, Expected=143.390
>Predicted=143.390, Expected=147.270
>Predicted=147.270, Expected=149.450
>Predicted=149.450, Expected=152.340
>Predicted=152.340, Expected=149.350
>Predicted=149.350, Expected=144.800
>Predicted=144.800, Expected=155.740
>Predicted=155.740, Expected=153.340
>Predicted=153.340, Expected=150.650
>Predicted=150.650, Expected=145.030
>Predicted=145.030, Expected=138.880
>Predicted=138.880, Expected=138.380
>Predicted=138.380, Expected=138.920
>Predicted=138.920, Expected=139.500
>Predicted=139.500, Expected=134.870
>Predicted=134.870, Expected=146.870
>Predicted=146.870, Expected=149.700
>Predicted=149.700, Expected=148.280
>Predicted=148.280, Expected=150.040
>Predicted=150.040, Expected=148.790
>Predicted=148.790, Expected=150.720
>Predicted=150.720, Expected=151.290
>Predicted=151.290, Expected=148.010
>Predicted=148.010, Expected=150.180
>Predicted=150.180, Expected=151.070
>Predicted=151.070, Expected=148.110
>Predicted=148.110, Expected=144.220
>Predicted=144.220, Expected=141.170
>Predicted=141.170, Expected=148.030
>Predicted=148.030, Expected=148.310
>Predicted=148.310, Expected=147.810
>Predicted=147.810, Expected=146.630
>Predicted=146.630, Expected=142.910
>Predicted=142.910, Expected=140.940
>Predicted=140.940, Expected=142.650
>Predicted=142.650, Expected=142.160
>Predicted=142.160, Expected=144.490
>Predicted=144.490, Expected=145.470
>Predicted=145.470, Expected=143.210
>Predicted=143.210, Expected=136.500
>Predicted=136.500, Expected=134.510
>Predicted=134.510, Expected=132.370
>Predicted=132.370, Expected=132.300
>Predicted=132.300, Expected=135.450
>Predicted=135.450, Expected=132.230
>Predicted=132.230, Expected=131.860
>Predicted=131.860, Expected=130.030
>Predicted=130.030, Expected=126.040
>Predicted=126.040, Expected=129.610
>Predicted=129.610, Expected=129.930
>Predicted=129.930, Expected=125.070
>Predicted=125.070, Expected=126.360
>Predicted=126.360, Expected=125.020
>Predicted=125.020, Expected=129.620
>Predicted=129.620, Expected=130.150
>Predicted=130.150, Expected=130.730
>Predicted=130.730, Expected=133.490
>Predicted=133.490, Expected=133.410
>Predicted=133.410, Expected=134.760
>Predicted=134.760, Expected=135.940
>Predicted=135.940, Expected=135.210
>Predicted=135.210, Expected=135.270
>Predicted=135.270, Expected=137.870
>Predicted=137.870, Expected=141.110
>Predicted=141.110, Expected=142.530
>Predicted=142.530, Expected=141.860
>Predicted=141.860, Expected=143.960
>Predicted=143.960, Expected=145.930
>Predicted=145.930, Expected=143.000
>Predicted=143.000, Expected=144.290
>Predicted=144.290, Expected=145.430
>Predicted=145.430, Expected=150.820
>Predicted=150.820, Expected=154.500
>Predicted=154.500, Expected=151.730
>Predicted=151.730, Expected=154.650
>Predicted=154.650, Expected=151.920
>Predicted=151.920, Expected=150.870
>Predicted=150.870, Expected=151.010
>Predicted=151.010, Expected=153.850
>Predicted=153.850, Expected=153.200
>Predicted=153.200, Expected=155.330
>Predicted=155.330, Expected=153.710
>Predicted=153.710, Expected=152.550
>Predicted=152.550, Expected=148.480
>Predicted=148.480, Expected=148.910
>Predicted=148.910, Expected=149.400
>Predicted=149.400, Expected=146.710
>Predicted=146.710, Expected=147.920
>Predicted=147.920, Expected=147.410
>Predicted=147.410, Expected=145.310
>Predicted=145.310, Expected=145.910
>Predicted=145.910, Expected=151.030
>Predicted=151.030, Expected=153.830
>Predicted=153.830, Expected=151.600
>Predicted=151.600, Expected=152.870
>Predicted=152.870, Expected=150.590
>Predicted=150.590, Expected=148.500
>Predicted=148.500, Expected=150.470
>Predicted=150.470, Expected=152.590
>Predicted=152.590, Expected=152.990
>Predicted=152.990, Expected=155.850
>Predicted=155.850, Expected=155.000
>Predicted=155.000, Expected=157.400
>Predicted=157.400, Expected=159.280
>Predicted=159.280, Expected=157.830
>Predicted=157.830, Expected=158.930
>Predicted=158.930, Expected=160.250
>Predicted=160.250, Expected=158.280
>Predicted=158.280, Expected=157.650
>Predicted=157.650, Expected=160.770
>Predicted=160.770, Expected=162.360
>Predicted=162.360, Expected=164.900
>Predicted=164.900, Expected=166.170
>Predicted=166.170, Expected=165.630
>Predicted=165.630, Expected=163.760
>Predicted=163.760, Expected=164.660
>Predicted=164.660, Expected=162.030
>Predicted=162.030, Expected=160.800
>Predicted=160.800, Expected=160.100
>Predicted=160.100, Expected=165.560
>Predicted=165.560, Expected=165.210
>Predicted=165.210, Expected=165.230
>Predicted=165.230, Expected=166.470
>Predicted=166.470, Expected=167.630
>Predicted=167.630, Expected=166.650
>Predicted=166.650, Expected=165.020
>Predicted=165.020, Expected=165.330
>Predicted=165.330, Expected=163.770
>Predicted=163.770, Expected=163.760
>Predicted=163.760, Expected=168.410
>Predicted=168.410, Expected=169.680
>Predicted=169.680, Expected=169.590
>Predicted=169.590, Expected=168.540
>Predicted=168.540, Expected=167.450
>Predicted=167.450, Expected=165.790
>Predicted=165.790, Expected=173.570
>Predicted=173.570, Expected=173.500
>Predicted=173.500, Expected=171.770
>Predicted=171.770, Expected=173.560
>Predicted=173.560, Expected=173.750
>Predicted=173.750, Expected=172.570
>Predicted=172.570, Expected=172.070
>Predicted=172.070, Expected=172.070
>Predicted=172.070, Expected=172.690
>Predicted=172.690, Expected=175.050
>Predicted=175.050, Expected=175.160
>Predicted=175.160, Expected=174.200
>Predicted=174.200, Expected=171.560
>Predicted=171.560, Expected=171.840
>Predicted=171.840, Expected=172.990
>Predicted=172.990, Expected=175.430
>Predicted=175.430, Expected=177.300
>Predicted=177.300, Expected=177.250
>Predicted=177.250, Expected=180.090
>Predicted=180.090, Expected=180.950
>Predicted=180.950, Expected=179.580
>Predicted=179.580, Expected=179.210
>Predicted=179.210, Expected=177.820
>Predicted=177.820, Expected=180.570
>Predicted=180.570, Expected=180.960
>Predicted=180.960, Expected=183.790
>Predicted=183.790, Expected=183.310
>Predicted=183.310, Expected=183.950
>Predicted=183.950, Expected=186.010
>Predicted=186.010, Expected=184.920
>Predicted=184.920, Expected=185.010
>Predicted=185.010, Expected=183.960
>Predicted=183.960, Expected=187.000
>Predicted=187.000, Expected=186.680
>Predicted=186.680, Expected=185.270
>Predicted=185.270, Expected=188.060
>Predicted=188.060, Expected=189.250
>Predicted=189.250, Expected=189.590
>Predicted=189.590, Expected=193.970
>Predicted=193.970, Expected=192.460
>Predicted=192.460, Expected=191.330
>Predicted=191.330, Expected=191.810
>Predicted=191.810, Expected=190.680
>Predicted=190.680, Expected=188.610
>Predicted=188.610, Expected=188.080
>Predicted=188.080, Expected=189.770
>Predicted=189.770, Expected=190.540
>Predicted=190.540, Expected=190.690
>Predicted=190.690, Expected=193.990
>Predicted=193.990, Expected=193.730
>Predicted=193.730, Expected=195.100
>Predicted=195.100, Expected=193.130
>Predicted=193.130, Expected=191.940
>Predicted=191.940, Expected=192.750
>Predicted=192.750, Expected=193.620
>Predicted=193.620, Expected=194.500
>Predicted=194.500, Expected=193.220
>Predicted=193.220, Expected=195.830
>Predicted=195.830, Expected=196.450
>Predicted=196.450, Expected=195.610
>Predicted=195.610, Expected=192.580
>Predicted=192.580, Expected=191.170
>Predicted=191.170, Expected=181.990
>Predicted=181.990, Expected=178.850
>Predicted=178.850, Expected=179.800
>Predicted=179.800, Expected=178.190
>Predicted=178.190, Expected=177.970
>Predicted=177.970, Expected=177.790
>Predicted=177.790, Expected=179.460
>Predicted=179.460, Expected=177.450
>Predicted=177.450, Expected=176.570
>Predicted=176.570, Expected=174.000
>Predicted=174.000, Expected=174.490
>Predicted=174.490, Expected=175.840
>Predicted=175.840, Expected=177.230
>Predicted=177.230, Expected=181.120
>Predicted=181.120, Expected=176.380
>Predicted=176.380, Expected=178.610
>Predicted=178.610, Expected=180.190
>Predicted=180.190, Expected=184.120
>Predicted=184.120, Expected=187.650
>Predicted=187.650, Expected=187.870
>Predicted=187.870, Expected=189.460
>Predicted=189.460, Expected=189.700
>Predicted=189.700, Expected=182.910
>Predicted=182.910, Expected=177.560
>Predicted=177.560, Expected=178.180
>Predicted=178.180, Expected=179.360
>Predicted=179.360, Expected=176.300
>Predicted=176.300, Expected=174.210
>Predicted=174.210, Expected=175.740
>Predicted=175.740, Expected=175.010
>Predicted=175.010, Expected=177.970
>Predicted=177.970, Expected=179.070
>Predicted=179.070, Expected=175.490
>Predicted=175.490, Expected=173.930
>Predicted=173.930, Expected=174.790
>Predicted=174.790, Expected=176.080
>Predicted=176.080, Expected=171.960
>Predicted=171.960, Expected=170.430
>Predicted=170.430, Expected=170.690
>Predicted=170.690, Expected=171.210
>Predicted=171.210, Expected=173.750
>Predicted=173.750, Expected=172.400
>Predicted=172.400, Expected=173.660
>Predicted=173.660, Expected=174.910
RMSE: 2.982

<ipython-input-7-c5f65b306fc0>:2: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True) # prepare data

# summary statistics of time series

series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
print(series.describe())

count    947.000000
mean     138.030993
std       32.288810
min       56.092499
25%      121.779999
50%      143.779999
75%      161.985001
max      196.449997
Name: 1, dtype: float64

<ipython-input-8-3512b164cf88>:3: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
series.plot()
pyplot.show()

<ipython-input-9-95d605ddc644>:3: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
pyplot.figure(1)
pyplot.subplot(211)
series.hist()
pyplot.subplot(212)
series.plot(kind='kde')
pyplot.show()

<ipython-input-10-25c0a3841568>:1: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# Load data
df = pd.read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True)

# Group the data by year
groups = df.groupby(pd.Grouper(freq='A'))

# Create a list to store data by year
years_data = []

# Populate the list with data grouped by year
for name, group in groups:
    years_data.append(group.values.flatten())

# Plot boxplots for each year
plt.figure(figsize=(15, 8))
plt.boxplot(years_data, labels=range(df.index.year.min(), df.index.year.max() + 1))
plt.title('Boxplots of Closing Prices by Year')
plt.xlabel('Year')
plt.ylabel('Closing Price')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()

# create a differenced time series
def difference(dataset):
  diff = list()
  for i in range(1, len(dataset)):
    value = dataset[i] - dataset[i - 1]
    diff.append(value)
  return Series(diff)
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
X = series.values
# difference data
stationary = difference(X)
stationary.index = series.index[1:]
# check if stationary
result = adfuller(stationary)
print('ADF Statistic: %f' % result[0])
print('p-value: %f' % result[1])
print('Critical Values:')
for key, value in result[4].items():

 print('\t%s: %.3f' % (key, value)) # save
stationary.to_csv('/content/drive/MyDrive/data/app_stationary.csv', header=False)

<ipython-input-13-3f7766769d09>:9: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

ADF Statistic: -32.042208
p-value: 0.000000
Critical Values:
	1%: -3.437
	5%: -2.865
	10%: -2.568

series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
pyplot.figure()
pyplot.subplot(211)
plot_acf(series, lags=50, ax=pyplot.gca())

pyplot.subplot(212)
plot_pacf(series, lags=50, ax=pyplot.gca())
pyplot.show()

<ipython-input-14-4a966e6d86af>:1: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]

# walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):

# predict
  model = ARIMA(history, order=(0,1,2))
  model_fit = model.fit()
  yhat = model_fit.forecast()[0]
  predictions.append(yhat)

  # observation
obs = test[i]
history.append(obs)
print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))

# report performance
rmse = sqrt(mean_squared_error(test, predictions))
print('RMSE: %.3f' % rmse)

<ipython-input-15-4218bb00964a>:2: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

>Predicted=150.052, Expected=174.910
RMSE: 19.768

# evaluate an ARIMA model for a given order (p,d,q) and return RMSE
def evaluate_arima_model(X, arima_order):
    # prepare training dataset
    X = X.astype('float32')
    train_size = int(len(X) * 0.50)
    train, test = X[0:train_size], X[train_size:]
    history = [x for x in train]

    # make predictions
    predictions = list()
    for t in range(len(test)):
        model = ARIMA(history, order=arima_order)
        model_fit = model.fit()
        yhat = model_fit.forecast()[0]
        predictions.append(yhat)
        history.append(test[t])

    # calculate out of sample error
    rmse = sqrt(mean_squared_error(test, predictions))
    return rmse

# evaluate combinations of p, d and q values for an ARIMA model
def evaluate_models(dataset, p_values, d_values, q_values):
    dataset = dataset.astype('float32')
    best_score, best_cfg = float("inf"), None
    for p in p_values:
        for d in d_values:
            for q in q_values:
                order = (p,d,q)
                try:
                    rmse = evaluate_arima_model(dataset, order)
                    if rmse < best_score:
                        best_score, best_cfg = rmse, order
                    print('ARIMA%s RMSE=%.3f' % (order, rmse))
                except:
                    continue
    print('Best ARIMA%s RMSE=%.3f' % (best_cfg, best_score))

# load dataset
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# evaluate parameters
p_values = range(0,2)
d_values = range(0, 3)
q_values = range(0, 2)
warnings.filterwarnings("ignore")
evaluate_models(series.values, p_values, d_values, q_values)

<ipython-input-5-589e6721015a>:40: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

ARIMA(0, 0, 0) RMSE=36.777
ARIMA(0, 0, 1) RMSE=19.132
ARIMA(0, 1, 0) RMSE=2.982
ARIMA(0, 1, 1) RMSE=2.995
ARIMA(0, 2, 0) RMSE=4.195
ARIMA(0, 2, 1) RMSE=2.985
ARIMA(1, 0, 0) RMSE=2.984
ARIMA(1, 0, 1) RMSE=2.996
ARIMA(1, 1, 0) RMSE=2.996
ARIMA(1, 1, 1) RMSE=2.999
ARIMA(1, 2, 0) RMSE=3.747
ARIMA(1, 2, 1) RMSE=3.000
Best ARIMA(0, 1, 0) RMSE=2.982

# plot residual errors for ARIMA model

# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
# walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):
# predict
  model = ARIMA(history, order=(0,1,0))
  model_fit = model.fit()
  yhat = model_fit.forecast()[0]
  predictions.append(yhat)
  # observation
  obs = test[i]
  history.append(obs)
# errors
residuals = [test[i]-predictions[i] for i in range(len(test))]
residuals = DataFrame(residuals)
pyplot.figure()
pyplot.subplot(211)
residuals.hist(ax=pyplot.gca())
pyplot.subplot(212)
residuals.plot(kind='kde', ax=pyplot.gca())
pyplot.show()

<ipython-input-3-c8f3bfbe71cd>:8: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# ACF and PACF plots of forecast residual errors

# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]
# walk-forward validation
history = [x for x in train]
predictions = list()


for i in range(len(test)):
  # predict
  model = ARIMA(history, order=(0,1,0))
  model_fit = model.fit()
  yhat = model_fit.forecast()[0]
  predictions.append(yhat)
  # observation
  obs = test[i]
  history.append(obs)
# errors
residuals = [test[i]-predictions[i] for i in range(len(test))]
residuals = DataFrame(residuals)
pyplot.figure()
pyplot.subplot(211)
plot_acf(residuals, lags=25, ax=pyplot.gca())
pyplot.subplot(212)
plot_pacf(residuals, lags=25, ax=pyplot.gca())
pyplot.show()

<ipython-input-4-6f3566148926>:9: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# plots of box-cox transformed dataset

series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
X = series.values
transformed, lam = boxcox(X)
print('Lambda: %f' % lam)
pyplot.figure(1)
# line plot
pyplot.subplot(311)
pyplot.plot(transformed)
# histogram
pyplot.subplot(312)


pyplot.hist(transformed)
# q-q plot
pyplot.subplot(313)
qqplot(transformed, line='r', ax=pyplot.gca())
pyplot.show()

<ipython-input-5-dfa7deb9ef96>:6: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

Lambda: 1.953080

# Function to invert Box-Cox transform
def boxcox_inverse(value, lam):
    if lam == 0:
        return exp(value)
    return exp(log(lam * value + 1) / lam)

# Load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

# Prepare data
X = series.values
X = X.astype('float32')
train_size = int(len(X) * 0.50)
train, test = X[0:train_size], X[train_size:]

# Walk-forward validation
history = [x for x in train]
predictions = list()
for i in range(len(test)):
    # Transform
    transformed, lam = boxcox(history)
    if lam < -5:
        transformed, lam = history, 1
    # Predict
    model = ARIMA(transformed, order=(0,1,0))
    model_fit = model.fit()
    yhat = model_fit.forecast()[0]
    # Invert transformed prediction
    yhat = boxcox_inverse(yhat, lam)
    predictions.append(yhat)
    # Observation
    obs = test[i]
    history.append(obs)
    print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))

# Report performance
rmse = sqrt(mean_squared_error(test, predictions))
print('RMSE: %.3f' % rmse)

<ipython-input-6-1d79d03f6b50>:16: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

>Predicted=150.000, Expected=151.000
>Predicted=151.000, Expected=153.490
>Predicted=153.490, Expected=157.870
>Predicted=157.870, Expected=160.550
>Predicted=160.550, Expected=161.020
>Predicted=161.020, Expected=161.410
>Predicted=161.410, Expected=161.940
>Predicted=161.940, Expected=156.810
>Predicted=156.810, Expected=160.240
>Predicted=160.240, Expected=165.300
>Predicted=165.300, Expected=164.770
>Predicted=164.770, Expected=163.760
>Predicted=163.760, Expected=161.840
>Predicted=161.840, Expected=165.320
>Predicted=165.320, Expected=171.180
>Predicted=171.180, Expected=175.080
>Predicted=175.080, Expected=174.560
>Predicted=174.560, Expected=179.450
>Predicted=179.450, Expected=175.740
>Predicted=175.740, Expected=174.330
>Predicted=174.330, Expected=179.300
>Predicted=179.300, Expected=172.260
>Predicted=172.260, Expected=171.140
>Predicted=171.140, Expected=169.750
>Predicted=169.750, Expected=172.990
>Predicted=172.990, Expected=175.640
>Predicted=175.640, Expected=176.280
>Predicted=176.280, Expected=180.330
>Predicted=180.330, Expected=179.290
>Predicted=179.290, Expected=179.380
>Predicted=179.380, Expected=178.200
>Predicted=178.200, Expected=177.570
>Predicted=177.570, Expected=182.010
>Predicted=182.010, Expected=179.700
>Predicted=179.700, Expected=174.920
>Predicted=174.920, Expected=172.000
>Predicted=172.000, Expected=172.170
>Predicted=172.170, Expected=172.190
>Predicted=172.190, Expected=175.080
>Predicted=175.080, Expected=175.530
>Predicted=175.530, Expected=172.190
>Predicted=172.190, Expected=173.070
>Predicted=173.070, Expected=169.800
>Predicted=169.800, Expected=166.230
>Predicted=166.230, Expected=164.510
>Predicted=164.510, Expected=162.410
>Predicted=162.410, Expected=161.620
>Predicted=161.620, Expected=159.780
>Predicted=159.780, Expected=159.690
>Predicted=159.690, Expected=159.220
>Predicted=159.220, Expected=170.330
>Predicted=170.330, Expected=174.780
>Predicted=174.780, Expected=174.610
>Predicted=174.610, Expected=175.840
>Predicted=175.840, Expected=172.900
>Predicted=172.900, Expected=172.390
>Predicted=172.390, Expected=171.660
>Predicted=171.660, Expected=174.830
>Predicted=174.830, Expected=176.280
>Predicted=176.280, Expected=172.120
>Predicted=172.120, Expected=168.640
>Predicted=168.640, Expected=168.880
>Predicted=168.880, Expected=172.790
>Predicted=172.790, Expected=172.550
>Predicted=172.550, Expected=168.880
>Predicted=168.880, Expected=167.300
>Predicted=167.300, Expected=164.320
>Predicted=164.320, Expected=160.070
>Predicted=160.070, Expected=162.740
>Predicted=162.740, Expected=164.850
>Predicted=164.850, Expected=165.120
>Predicted=165.120, Expected=163.200
>Predicted=163.200, Expected=166.560
>Predicted=166.560, Expected=166.230
>Predicted=166.230, Expected=163.170
>Predicted=163.170, Expected=159.300
>Predicted=159.300, Expected=157.440
>Predicted=157.440, Expected=162.950
>Predicted=162.950, Expected=158.520
>Predicted=158.520, Expected=154.730
>Predicted=154.730, Expected=150.620
>Predicted=150.620, Expected=155.090
>Predicted=155.090, Expected=159.590
>Predicted=159.590, Expected=160.620
>Predicted=160.620, Expected=163.980
>Predicted=163.980, Expected=165.380
>Predicted=165.380, Expected=168.820
>Predicted=168.820, Expected=170.210
>Predicted=170.210, Expected=174.070
>Predicted=174.070, Expected=174.720
>Predicted=174.720, Expected=175.600
>Predicted=175.600, Expected=178.960
>Predicted=178.960, Expected=177.770
>Predicted=177.770, Expected=174.610
>Predicted=174.610, Expected=174.310
>Predicted=174.310, Expected=178.440
>Predicted=178.440, Expected=175.060
>Predicted=175.060, Expected=171.830
>Predicted=171.830, Expected=172.140
>Predicted=172.140, Expected=170.090
>Predicted=170.090, Expected=165.750
>Predicted=165.750, Expected=167.660
>Predicted=167.660, Expected=170.400
>Predicted=170.400, Expected=165.290
>Predicted=165.290, Expected=165.070
>Predicted=165.070, Expected=167.400
>Predicted=167.400, Expected=167.230
>Predicted=167.230, Expected=166.420
>Predicted=166.420, Expected=161.790
>Predicted=161.790, Expected=162.880
>Predicted=162.880, Expected=156.800
>Predicted=156.800, Expected=156.570
>Predicted=156.570, Expected=163.640
>Predicted=163.640, Expected=157.650
>Predicted=157.650, Expected=157.960
>Predicted=157.960, Expected=159.480
>Predicted=159.480, Expected=166.020
>Predicted=166.020, Expected=156.770
>Predicted=156.770, Expected=157.280
>Predicted=157.280, Expected=152.060
>Predicted=152.060, Expected=154.510
>Predicted=154.510, Expected=146.500
>Predicted=146.500, Expected=142.560
>Predicted=142.560, Expected=147.110
>Predicted=147.110, Expected=145.540
>Predicted=145.540, Expected=149.240
>Predicted=149.240, Expected=140.820
>Predicted=140.820, Expected=137.350
>Predicted=137.350, Expected=137.590
>Predicted=137.590, Expected=143.110
>Predicted=143.110, Expected=140.360
>Predicted=140.360, Expected=140.520
>Predicted=140.520, Expected=143.780
>Predicted=143.780, Expected=149.640
>Predicted=149.640, Expected=148.840
>Predicted=148.840, Expected=148.710
>Predicted=148.710, Expected=151.210
>Predicted=151.210, Expected=145.380
>Predicted=145.380, Expected=146.140
>Predicted=146.140, Expected=148.710
>Predicted=148.710, Expected=147.960
>Predicted=147.960, Expected=142.640
>Predicted=142.640, Expected=137.130
>Predicted=137.130, Expected=131.880
>Predicted=131.880, Expected=132.760
>Predicted=132.760, Expected=135.430
>Predicted=135.430, Expected=130.060
>Predicted=130.060, Expected=131.560
>Predicted=131.560, Expected=135.870
>Predicted=135.870, Expected=135.350
>Predicted=135.350, Expected=138.270
>Predicted=138.270, Expected=141.660
>Predicted=141.660, Expected=141.660
>Predicted=141.660, Expected=137.440
>Predicted=137.440, Expected=139.230
>Predicted=139.230, Expected=136.720
>Predicted=136.720, Expected=138.930
>Predicted=138.930, Expected=141.560
>Predicted=141.560, Expected=142.920
>Predicted=142.920, Expected=146.350
>Predicted=146.350, Expected=147.040
>Predicted=147.040, Expected=144.870
>Predicted=144.870, Expected=145.860
>Predicted=145.860, Expected=145.490
>Predicted=145.490, Expected=148.470
>Predicted=148.470, Expected=150.170
>Predicted=150.170, Expected=147.070
>Predicted=147.070, Expected=151.000
>Predicted=151.000, Expected=153.040
>Predicted=153.040, Expected=155.350
>Predicted=155.350, Expected=154.090
>Predicted=154.090, Expected=152.950
>Predicted=152.950, Expected=151.600
>Predicted=151.600, Expected=156.790
>Predicted=156.790, Expected=157.350
>Predicted=157.350, Expected=162.510
>Predicted=162.510, Expected=161.510
>Predicted=161.510, Expected=160.010
>Predicted=160.010, Expected=166.130
>Predicted=166.130, Expected=165.810
>Predicted=165.810, Expected=165.350
>Predicted=165.350, Expected=164.870
>Predicted=164.870, Expected=164.920
>Predicted=164.920, Expected=169.240
>Predicted=169.240, Expected=168.490
>Predicted=168.490, Expected=172.100
>Predicted=172.100, Expected=173.190
>Predicted=173.190, Expected=173.030
>Predicted=173.030, Expected=174.550
>Predicted=174.550, Expected=174.150
>Predicted=174.150, Expected=171.520
>Predicted=171.520, Expected=167.570
>Predicted=167.570, Expected=167.230
>Predicted=167.230, Expected=167.530
>Predicted=167.530, Expected=170.030
>Predicted=170.030, Expected=163.620
>Predicted=163.620, Expected=161.380
>Predicted=161.380, Expected=158.910
>Predicted=158.910, Expected=157.220
>Predicted=157.220, Expected=157.960
>Predicted=157.960, Expected=155.810
>Predicted=155.810, Expected=154.530
>Predicted=154.530, Expected=155.960
>Predicted=155.960, Expected=154.460
>Predicted=154.460, Expected=157.370
>Predicted=157.370, Expected=163.430
>Predicted=163.430, Expected=153.840
>Predicted=153.840, Expected=155.310
>Predicted=155.310, Expected=152.370
>Predicted=152.370, Expected=150.700
>Predicted=150.700, Expected=154.480
>Predicted=154.480, Expected=156.900
>Predicted=156.900, Expected=153.720
>Predicted=153.720, Expected=152.740
>Predicted=152.740, Expected=150.430
>Predicted=150.430, Expected=150.770
>Predicted=150.770, Expected=151.760
>Predicted=151.760, Expected=149.840
>Predicted=149.840, Expected=142.480
>Predicted=142.480, Expected=138.200
>Predicted=138.200, Expected=142.450
>Predicted=142.450, Expected=146.100
>Predicted=146.100, Expected=146.400
>Predicted=146.400, Expected=145.430
>Predicted=145.430, Expected=140.090
>Predicted=140.090, Expected=140.420
>Predicted=140.420, Expected=138.980
>Predicted=138.980, Expected=138.340
>Predicted=138.340, Expected=142.990
>Predicted=142.990, Expected=138.380
>Predicted=138.380, Expected=142.410
>Predicted=142.410, Expected=143.750
>Predicted=143.750, Expected=143.860
>Predicted=143.860, Expected=143.390
>Predicted=143.390, Expected=147.270
>Predicted=147.270, Expected=149.450
>Predicted=149.450, Expected=152.340
>Predicted=152.340, Expected=149.350
>Predicted=149.350, Expected=144.800
>Predicted=144.800, Expected=155.740
>Predicted=155.740, Expected=153.340
>Predicted=153.340, Expected=150.650
>Predicted=150.650, Expected=145.030
>Predicted=145.030, Expected=138.880
>Predicted=138.880, Expected=138.380
>Predicted=138.380, Expected=138.920
>Predicted=138.920, Expected=139.500
>Predicted=139.500, Expected=134.870
>Predicted=134.870, Expected=146.870
>Predicted=146.870, Expected=149.700
>Predicted=149.700, Expected=148.280
>Predicted=148.280, Expected=150.040
>Predicted=150.040, Expected=148.790
>Predicted=148.790, Expected=150.720
>Predicted=150.720, Expected=151.290
>Predicted=151.290, Expected=148.010
>Predicted=148.010, Expected=150.180
>Predicted=150.180, Expected=151.070
>Predicted=151.070, Expected=148.110
>Predicted=148.110, Expected=144.220
>Predicted=144.220, Expected=141.170
>Predicted=141.170, Expected=148.030
>Predicted=148.030, Expected=148.310
>Predicted=148.310, Expected=147.810
>Predicted=147.810, Expected=146.630
>Predicted=146.630, Expected=142.910
>Predicted=142.910, Expected=140.940
>Predicted=140.940, Expected=142.650
>Predicted=142.650, Expected=142.160
>Predicted=142.160, Expected=144.490
>Predicted=144.490, Expected=145.470
>Predicted=145.470, Expected=143.210
>Predicted=143.210, Expected=136.500
>Predicted=136.500, Expected=134.510
>Predicted=134.510, Expected=132.370
>Predicted=132.370, Expected=132.300
>Predicted=132.300, Expected=135.450
>Predicted=135.450, Expected=132.230
>Predicted=132.230, Expected=131.860
>Predicted=131.860, Expected=130.030
>Predicted=130.030, Expected=126.040
>Predicted=126.040, Expected=129.610
>Predicted=129.610, Expected=129.930
>Predicted=129.930, Expected=125.070
>Predicted=125.070, Expected=126.360
>Predicted=126.360, Expected=125.020
>Predicted=125.020, Expected=129.620
>Predicted=129.620, Expected=130.150
>Predicted=130.150, Expected=130.730
>Predicted=130.730, Expected=133.490
>Predicted=133.490, Expected=133.410
>Predicted=133.410, Expected=134.760
>Predicted=134.760, Expected=135.940
>Predicted=135.940, Expected=135.210
>Predicted=135.210, Expected=135.270
>Predicted=135.270, Expected=137.870
>Predicted=137.870, Expected=141.110
>Predicted=141.110, Expected=142.530
>Predicted=142.530, Expected=141.860
>Predicted=141.860, Expected=143.960
>Predicted=143.960, Expected=145.930
>Predicted=145.930, Expected=143.000
>Predicted=143.000, Expected=144.290
>Predicted=144.290, Expected=145.430
>Predicted=145.430, Expected=150.820
>Predicted=150.820, Expected=154.500
>Predicted=154.500, Expected=151.730
>Predicted=151.730, Expected=154.650
>Predicted=154.650, Expected=151.920
>Predicted=151.920, Expected=150.870
>Predicted=150.870, Expected=151.010
>Predicted=151.010, Expected=153.850
>Predicted=153.850, Expected=153.200
>Predicted=153.200, Expected=155.330
>Predicted=155.330, Expected=153.710
>Predicted=153.710, Expected=152.550
>Predicted=152.550, Expected=148.480
>Predicted=148.480, Expected=148.910
>Predicted=148.910, Expected=149.400
>Predicted=149.400, Expected=146.710
>Predicted=146.710, Expected=147.920
>Predicted=147.920, Expected=147.410
>Predicted=147.410, Expected=145.310
>Predicted=145.310, Expected=145.910
>Predicted=145.910, Expected=151.030
>Predicted=151.030, Expected=153.830
>Predicted=153.830, Expected=151.600
>Predicted=151.600, Expected=152.870
>Predicted=152.870, Expected=150.590
>Predicted=150.590, Expected=148.500
>Predicted=148.500, Expected=150.470
>Predicted=150.470, Expected=152.590
>Predicted=152.590, Expected=152.990
>Predicted=152.990, Expected=155.850
>Predicted=155.850, Expected=155.000
>Predicted=155.000, Expected=157.400
>Predicted=157.400, Expected=159.280
>Predicted=159.280, Expected=157.830
>Predicted=157.830, Expected=158.930
>Predicted=158.930, Expected=160.250
>Predicted=160.250, Expected=158.280
>Predicted=158.280, Expected=157.650
>Predicted=157.650, Expected=160.770
>Predicted=160.770, Expected=162.360
>Predicted=162.360, Expected=164.900
>Predicted=164.900, Expected=166.170
>Predicted=166.170, Expected=165.630
>Predicted=165.630, Expected=163.760
>Predicted=163.760, Expected=164.660
>Predicted=164.660, Expected=162.030
>Predicted=162.030, Expected=160.800
>Predicted=160.800, Expected=160.100
>Predicted=160.100, Expected=165.560
>Predicted=165.560, Expected=165.210
>Predicted=165.210, Expected=165.230
>Predicted=165.230, Expected=166.470
>Predicted=166.470, Expected=167.630
>Predicted=167.630, Expected=166.650
>Predicted=166.650, Expected=165.020
>Predicted=165.020, Expected=165.330
>Predicted=165.330, Expected=163.770
>Predicted=163.770, Expected=163.760
>Predicted=163.760, Expected=168.410
>Predicted=168.410, Expected=169.680
>Predicted=169.680, Expected=169.590
>Predicted=169.590, Expected=168.540
>Predicted=168.540, Expected=167.450
>Predicted=167.450, Expected=165.790
>Predicted=165.790, Expected=173.570
>Predicted=173.570, Expected=173.500
>Predicted=173.500, Expected=171.770
>Predicted=171.770, Expected=173.560
>Predicted=173.560, Expected=173.750
>Predicted=173.750, Expected=172.570
>Predicted=172.570, Expected=172.070
>Predicted=172.070, Expected=172.070
>Predicted=172.070, Expected=172.690
>Predicted=172.690, Expected=175.050
>Predicted=175.050, Expected=175.160
>Predicted=175.160, Expected=174.200
>Predicted=174.200, Expected=171.560
>Predicted=171.560, Expected=171.840
>Predicted=171.840, Expected=172.990
>Predicted=172.990, Expected=175.430
>Predicted=175.430, Expected=177.300
>Predicted=177.300, Expected=177.250
>Predicted=177.250, Expected=180.090
>Predicted=180.090, Expected=180.950
>Predicted=180.950, Expected=179.580
>Predicted=179.580, Expected=179.210
>Predicted=179.210, Expected=177.820
>Predicted=177.820, Expected=180.570
>Predicted=180.570, Expected=180.960
>Predicted=180.960, Expected=183.790
>Predicted=183.790, Expected=183.310
>Predicted=183.310, Expected=183.950
>Predicted=183.950, Expected=186.010
>Predicted=186.010, Expected=184.920
>Predicted=184.920, Expected=185.010
>Predicted=185.010, Expected=183.960
>Predicted=183.960, Expected=187.000
>Predicted=187.000, Expected=186.680
>Predicted=186.680, Expected=185.270
>Predicted=185.270, Expected=188.060
>Predicted=188.060, Expected=189.250
>Predicted=189.250, Expected=189.590
>Predicted=189.590, Expected=193.970
>Predicted=193.970, Expected=192.460
>Predicted=192.460, Expected=191.330
>Predicted=191.330, Expected=191.810
>Predicted=191.810, Expected=190.680
>Predicted=190.680, Expected=188.610
>Predicted=188.610, Expected=188.080
>Predicted=188.080, Expected=189.770
>Predicted=189.770, Expected=190.540
>Predicted=190.540, Expected=190.690
>Predicted=190.690, Expected=193.990
>Predicted=193.990, Expected=193.730
>Predicted=193.730, Expected=195.100
>Predicted=195.100, Expected=193.130
>Predicted=193.130, Expected=191.940
>Predicted=191.940, Expected=192.750
>Predicted=192.750, Expected=193.620
>Predicted=193.620, Expected=194.500
>Predicted=194.500, Expected=193.220
>Predicted=193.220, Expected=195.830
>Predicted=195.830, Expected=196.450
>Predicted=196.450, Expected=195.610
>Predicted=195.610, Expected=192.580
>Predicted=192.580, Expected=191.170
>Predicted=191.170, Expected=181.990
>Predicted=181.990, Expected=178.850
>Predicted=178.850, Expected=179.800
>Predicted=179.800, Expected=178.190
>Predicted=178.190, Expected=177.970
>Predicted=177.970, Expected=177.790
>Predicted=177.790, Expected=179.460
>Predicted=179.460, Expected=177.450
>Predicted=177.450, Expected=176.570
>Predicted=176.570, Expected=174.000
>Predicted=174.000, Expected=174.490
>Predicted=174.490, Expected=175.840
>Predicted=175.840, Expected=177.230
>Predicted=177.230, Expected=181.120
>Predicted=181.120, Expected=176.380
>Predicted=176.380, Expected=178.610
>Predicted=178.610, Expected=180.190
>Predicted=180.190, Expected=184.120
>Predicted=184.120, Expected=187.650
>Predicted=187.650, Expected=187.870
>Predicted=187.870, Expected=189.460
>Predicted=189.460, Expected=189.700
>Predicted=189.700, Expected=182.910
>Predicted=182.910, Expected=177.560
>Predicted=177.560, Expected=178.180
>Predicted=178.180, Expected=179.360
>Predicted=179.360, Expected=176.300
>Predicted=176.300, Expected=174.210
>Predicted=174.210, Expected=175.740
>Predicted=175.740, Expected=175.010
>Predicted=175.010, Expected=177.970
>Predicted=177.970, Expected=179.070
>Predicted=179.070, Expected=175.490
>Predicted=175.490, Expected=173.930
>Predicted=173.930, Expected=174.790
>Predicted=174.790, Expected=176.080
>Predicted=176.080, Expected=171.960
>Predicted=171.960, Expected=170.430
>Predicted=170.430, Expected=170.690
>Predicted=170.690, Expected=171.210
>Predicted=171.210, Expected=173.750
>Predicted=173.750, Expected=172.400
>Predicted=172.400, Expected=173.660
>Predicted=173.660, Expected=174.910
RMSE: 2.982

# finalize model and save to file

# load data
series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
# prepare data
X = series.values
X = X.astype('float32')
# transform data
transformed, lam = boxcox(X)
# fit model
model = ARIMA(transformed, order=(0,1,0))
model_fit = model.fit()
# save model
model_fit.save('/content/drive/MyDrive/data/app_model.pkl')
numpy.save('/content/drive/MyDrive/data/app_model_lambda.npy', [lam])

<ipython-input-7-1cc29c08cf7c>:7: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  series = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)

Predictions

# load the finalized model and make a prediction

# invert box-cox transform
def boxcox_inverse(value, lam):

  if lam == 0:
    return exp(value)
  return exp(log(lam * value + 1) / lam)
model_fit = ARIMAResults.load('/content/drive/MyDrive/data/app_model.pkl')
lam = numpy.load('/content/drive/MyDrive/data/app_model_lambda.npy')
yhat = model_fit.forecast()[0]
yhat = boxcox_inverse(yhat, lam)
print('Predicted: %.3f' % yhat)

Predicted: 174.910

<ipython-input-8-8fed440f2b22>:11: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
  return exp(log(lam * value + 1) / lam)

# invert box-cox transform
def boxcox_inverse(value, lam):
    if lam == 0:
        return exp(value)
    return exp(log(lam * value + 1) / lam)

# load and prepare datasets
dataset = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
X = dataset.values.astype('float32')
history = [x for x in X]
validation = read_csv('/content/drive/MyDrive/data/app_validation.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
y = validation.values.astype('float32')

# load model
model_fit = ARIMAResults.load('/content/drive/MyDrive/data/app_model.pkl')
lam = numpy.load('/content/drive/MyDrive/data/app_model_lambda.npy')

# make first prediction
predictions = list()
yhat = model_fit.forecast()[0]
yhat = boxcox_inverse(yhat, lam)
predictions.append(yhat)
history.append(y[0])
print('>Predicted=%.3f, Expected=%.3f' % (yhat, y[0]))

# rolling forecasts
for i in range(1, len(y)):
    # transform
    transformed, lam = boxcox(history)
    if lam < -5:
        transformed, lam = history, 1
    # predict
    model = ARIMA(transformed, order=(0,1,0))
    model_fit = model.fit()
    yhat = model_fit.forecast()[0]
    # invert transformed prediction
    yhat = boxcox_inverse(yhat, lam)
    predictions.append(yhat)
    # observation
    obs = y[i]
    history.append(obs)
    print('>Predicted=%.3f, Expected=%.3f' % (yhat, obs))

# report performance
rmse = sqrt(mean_squared_error(y, predictions))
print('RMSE: %.3f' % rmse)

pyplot.plot(y)
pyplot.plot(predictions, color='red')
pyplot.show()

<ipython-input-9-89857cb596e9>:19: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  dataset = read_csv('/content/drive/MyDrive/data/app_dataset.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
<ipython-input-9-89857cb596e9>:22: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.


  validation = read_csv('/content/drive/MyDrive/data/app_validation.csv', header=None, index_col=0, parse_dates=True, squeeze=True)
<ipython-input-9-89857cb596e9>:16: DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
  return exp(log(lam * value + 1) / lam)

>Predicted=174.910, Expected=177.490
>Predicted=177.490, Expected=178.990
>Predicted=178.990, Expected=178.390
>Predicted=178.390, Expected=179.800
>Predicted=179.800, Expected=180.710
>Predicted=180.710, Expected=178.850
>Predicted=178.850, Expected=178.720
>Predicted=178.720, Expected=177.150
>Predicted=177.150, Expected=175.840
>Predicted=175.840, Expected=175.460
>Predicted=175.460, Expected=172.880
>Predicted=172.880, Expected=173.000
>Predicted=173.000, Expected=173.440
>Predicted=173.440, Expected=171.100
>Predicted=171.100, Expected=166.890
>Predicted=166.890, Expected=168.220
>Predicted=168.220, Expected=170.290
>Predicted=170.290, Expected=170.770
>Predicted=170.770, Expected=173.970
>Predicted=173.970, Expected=177.570
>Predicted=177.570, Expected=176.650
>Predicted=176.650, Expected=179.230
>Predicted=179.230, Expected=181.820
>Predicted=181.820, Expected=182.890
>Predicted=182.890, Expected=182.410
>Predicted=182.410, Expected=186.400
>Predicted=186.400, Expected=184.800

/usr/local/lib/python3.10/dist-packages/numpy/core/_methods.py:187: RuntimeWarning: overflow encountered in reduce
  ret = umr_sum(x, axis, dtype, out, keepdims=keepdims, where=where)

>Predicted=184.800, Expected=187.440
>Predicted=187.440, Expected=188.010
>Predicted=188.010, Expected=189.710
>Predicted=189.710, Expected=189.690
>Predicted=189.690, Expected=191.450
>Predicted=191.450, Expected=190.640
>Predicted=190.640, Expected=191.310
>Predicted=191.310, Expected=189.970
>Predicted=189.970, Expected=189.790
>Predicted=189.790, Expected=190.400
>Predicted=190.400, Expected=189.370
>Predicted=189.370, Expected=189.950
>Predicted=189.950, Expected=191.240
>Predicted=191.240, Expected=189.430
>Predicted=189.430, Expected=193.420
>Predicted=193.420, Expected=192.320
>Predicted=192.320, Expected=194.270
>Predicted=194.270, Expected=195.710
>Predicted=195.710, Expected=193.180
>Predicted=193.180, Expected=194.710
>Predicted=194.710, Expected=197.960
>Predicted=197.960, Expected=198.110
>Predicted=198.110, Expected=197.570
>Predicted=197.570, Expected=195.890
>Predicted=195.890, Expected=196.940
>Predicted=196.940, Expected=194.830
>Predicted=194.830, Expected=194.680
>Predicted=194.680, Expected=193.600
>Predicted=193.600, Expected=193.050
>Predicted=193.050, Expected=193.150
>Predicted=193.150, Expected=193.580
>Predicted=193.580, Expected=192.530
>Predicted=192.530, Expected=185.640
>Predicted=185.640, Expected=184.250
>Predicted=184.250, Expected=181.910
>Predicted=181.910, Expected=181.180
>Predicted=181.180, Expected=185.560
>Predicted=185.560, Expected=185.140
>Predicted=185.140, Expected=186.190
>Predicted=186.190, Expected=185.590
>Predicted=185.590, Expected=185.920
>Predicted=185.920, Expected=183.630
>Predicted=183.630, Expected=182.680
>Predicted=182.680, Expected=188.630
>Predicted=188.630, Expected=191.560
>Predicted=191.560, Expected=193.890
>Predicted=193.890, Expected=195.180
>Predicted=195.180, Expected=194.500
>Predicted=194.500, Expected=194.170
>Predicted=194.170, Expected=192.420
>Predicted=192.420, Expected=191.730
>Predicted=191.730, Expected=188.040
>Predicted=188.040, Expected=184.400
>Predicted=184.400, Expected=186.860
>Predicted=186.860, Expected=185.850
>Predicted=185.850, Expected=187.680
>Predicted=187.680, Expected=189.300
>Predicted=189.300, Expected=189.410
>Predicted=189.410, Expected=188.320
>Predicted=188.320, Expected=188.850
>Predicted=188.850, Expected=187.150
>Predicted=187.150, Expected=185.040
>Predicted=185.040, Expected=184.150
>Predicted=184.150, Expected=183.860
>Predicted=183.860, Expected=182.310
>Predicted=182.310, Expected=181.560
>Predicted=181.560, Expected=182.320
>Predicted=182.320, Expected=184.370
>Predicted=184.370, Expected=182.520
>Predicted=182.520, Expected=181.160
>Predicted=181.160, Expected=182.630
>Predicted=182.630, Expected=181.420
>Predicted=181.420, Expected=180.750
RMSE: 1.986

Team Contribution

Name	Tasks	Time Spent
Hemraj Yadav	Load Data, Vizualization, EDA, Decomposition, TS Model, Prediction	40 hours
Amit Sai Jitta	Time Series Decomposition, TS Model, Prediction	25 hours

References

More information on the fasteda package.

Yahoo Finance: https://en.wikipedia.org/wiki/List_of_S%26P_500_companies

https://www.kaggle.com/code/faressayah/stock-market-analysis-prediction-using-lstm

https://plotly.com/python/templates/

Coding practices from class mainly 10 and 11

from google.colab import drive
import nbconvert

# Mount Google Drive to access files
drive.mount('/content/drive')

# Provide the path to your notebook file
notebook_path = '/content/drive/MyDrive/Colab Notebooks/Group11_Final_Project_Part_2_TMSP_V2.ipynb'  # Replace with your actual notebook path

# Convert notebook to HTML
html_exporter = nbconvert.HTMLExporter()
(body, resources) = html_exporter.from_filename(notebook_path)

# Write HTML content to a file
html_file_path = '/content/drive/MyDrive/Colab Notebooks/Group11_Final_Project_Part_2_TMSP_V2.html'  # Replace with your desired HTML file path
with open(html_file_path, 'w') as f:
    f.write(body)